INFORMATICA | The Data Integration Company

The main goals of any Application ILM project are to reduce cost, improve application performance while maintaining compliance.  To meet these goals, data has to be moved from a production database to either an online, accessible archive or purged completely from the system.  In either case, data is deleted from production.  Deleting data can have a significant impact on the production system’s performance if not executed carefully.  However, once the data is gone – the benefits have a ripple effect.  Production tables are smaller and more manageable.  Recovery times and maintenance windows can be reduced.  The cost of managing a smaller production database is reduced proportionally with the amount of data removed. 

In order to realize the true benefits of a reduced database, the DBA’s will need to participate in a few cleanup activities – possibly reorganizing the database to reclaim the space.  When data is selectively removed from a database, the storage of the remaining information resembles Swiss cheese.  Reorganizing the database may or may not be a significant effort – regardless, the storage benefit will not be realized unless the space can be released at the file system.   Once the DBA’s clean up the database, the storage team may also need to participate in a series of reorganization tasks within the storage array.    These last two steps cannot be overlooked.  There will be no benefit to an ILM project if the storage cannot be freed and reused for another application or released and credited in a charge-back model.  The initial measured benefit of the project will be the amount of consumed resources before the ILM project minus the amount of consumed resources after the cleanup activities.

An Application ILM project can provide immediate relief when the storage has been released.  However, a more important aspect of an ILM project is to evaluate the behaviors that got you here to begin with.  ILM is not a technology problem, but rather a behavior problem.  Once you can identify the broken processes or find the missing checks and balances in the consumption of IT resources, make the corrections, start rewarding responsible behaviors and integrate ILM activities into your day to day operations.

Julie Lockner  President & Founder  www.CentricInfo.com

The cover story on the March 8th issue of Computerworld is “Swinging Toward Centralization”. It talks about the pendulum moving toward IT consolidation as organizations strive to save money and improve controls. This article is not alone. A growing number of analysts are talking about consolidation, Lean IT practices, and taking “a production line approach to integration”.  When you combine all this rhetoric with the industry fundamentals, the stage is set for 2010 to The Year of the ICC.

By this I mean that 2010 will be the year when the pace of Integration Competency Center implementations picks up speed and we end the year with a majority of large organizations having one or more competency centers (or COE’s).  Gartner predicted this back in 2006, and the industry is on track to prove the prediction right.

The Computerworld article goes on to say that “37% of 312 respondents said that it’s likely that their IT units will become more centralized in the next 12 months.” A recent independent research (1) report notes “…application development professionals and enterprise architects should consolidate their skilled application, process, and data integration resources into a cross-enterprise shared services organization that will deliver faster, more economical integration solutions to support cross-enterprise needs.”

The other reasons that I believe 2010 will be the Year of the ICC include:

1. While the recession is “technically” over, IT organizations are not out of the woods and the pressure to contain costs is as high as ever.
2. The macro trends of globalization, rapid growth through acquisition, increased regulatory requirements, and proactive risk management are continuing unabated.
3. The proliferation (explosion in some cases) of data has reached a critical level in many enterprises whereby the costs and risks of unmanaged information silos demands immediate attention and can’t be deferred to future years.
4. The Lean Integration book (see www.integrationfactory.com) will be published next month. This is yet another example of the growing body of knowledge in the industry for how to effectively, and successfully, implement an ICC.
5. The Informatica 9 Platform provides a step-function increase in capability which enables organizations to dramatically enhance and simplify their integration infrastructure.
6. Professional services organizations such as Informatica, Accenture, Wipro, HP, and Teradata offer a broad range of services based on leading practices.

Items 1-3 explain WHY organizations need an ICC and items 4-6 show HOW to successfully implement an enterprise-wide integration strategy. 2010 is a “perfect storm” of motive and means which is why this will be The Year of the ICC.

Notes: (1)  “Update: Evaluating Integration Alternatives,” Forrester Research, Inc., January, 2010

Good architecture is the foundation required for agile systems that are responsive to business needs. We see that with SOA, which has been missing a vital piece of its architectural approach – from the data. Since it came about in its current form in the early-to-mid 2000s, service-orientation (mainly focused on applications) has existed in a separate world from data management.

Problem is, an SOA-enabled infrastructure with bad data flowing through it can be useless and even dangerous. One commentator even compared SOA to a mosquito that can deliver payloads of bad data (”viral data”) at lightning speed all across the enterprise — pandemic style — before it can be stopped.

The recent launch of the SOA Data Integration Architect Community (SDIAC), an online community focused on the value of data integration and data services in agile architectures such as SOA, promises to help bring the data and SOA worlds get closer together. The organization will bring support to SOA proponents struggling to address data quality issues and data architects seeking to service-enable their environments.

But what will the role of the data warehouse be in a data-driven service oriented architecture? Where and how will these behemoths fit? Seeking to address this piece of the architecture puzzle, long-time data warehousing expert Barry Devlin observes that the role of the traditional data warehouse has been supplemented to some degree by initiatives such as SOA and Web/Enterprise 2.0.

More so than data warehouses, SOA and Web/Enterprise 2.0 are re-arranging the relationships and boundaries between operational, informational, and collaborative systems. They bring in data from various sources across the enterprise, and provide new ways for end-users to manipulate and mine the information. He notes that SOA, for example, is bringing modular, plug-and-play approaches into the equation, giving rise to possibilities of new types of analysis applications being integrated into various enterprise applications.

The data warehouse approach fell out of vogue, as it became untenable to bring together so much information exploding all across the enterprise into a single, centralized database.

Devlin urges that this model be revisited, however – in an updated form. He observes, in line with Occam's razor-style thinking, that a data warehouse architecture is the simplest solution to an increasingly complex issue:

“The simpler the answer, the better the solution. If you want to create a consistent, integrated information resource, you must stop creating duplicates of existing information that have to be managed to consistency, and you must eliminate, or substantially reduce, existing data duplication. The original data warehouse architecture did this. It proposed a logically single data store – the business data warehouse – modeled at the enterprise level as the consistent and integrated source of all information for decision making. This simplicity was ultimately lost with the emergence of the layered architecture, due to a combination of database performance and enterprise modeling issues.”

Devlin proposes an architectural approach he calls “Business Integrated Insight” (or BI 2). This is an architecture that “gathers all the information of the enterprise, hard and soft, operational, informational and collaborative into a single logical component.” A second integration point consists of “a logically single, consistent and integrated set of processes that spans all aspects of business and IT needs is required to enable the flexibility and adaptability that modern business requires.”

Devlin also posted a white paper explaining Business Integrated Insight architecture in more detail.

Essentially, Devlin proposes a three-tier architecture:

• Layer 1 – Business Information Resource: This is the single foundational layer with all information —operational, informational and collaborative information, as well as metadata.
• Layer 2 – Business Function Assembly: “The middle layer consists of processes, workflows, tasks, applications, tools and so on: in short, all functionality and processing that runs on the business’ computers.”
• Layer 3 – Personal Action Domain: This is the interface layer to all business users within the company.

The value of such a simplified view is to provide an new architected approach to support the demands on businesses that  are highly interlinked and interdependent, with entwined processes and pressured reaction times, Devlin says. “Business and IT, in particular, need to take a fully-integrated view of what is required from its IT environment.”

While it's uncertain whether Devlin's proposal will catch on, he has pinpointed one of the greatest challenges in bringing together multiple initiatives across the enterprise, involving applications and end-user interfaces. Effective data management can't happen in a vacuum, and enterprises are begging for simpler, more straightforward architectural solutions to out-of-control complexity.

Loraine Lawson did a great job covering the topic of the integration challenges around the cloud and virtualization. She reports that “…a recent Internet Evolution column [by David Vellante] looks more broadly at the cloud integration question and concludes that insufficient integration is holding up both cloud computing and virtualization.”

In fact, what currently limits the number of cloud deployments is the lack of a clear understanding of data integration in the context of cloud computing. This is a rather easy problem to solve, but it’s often an afterthought.

The core issue is that cloud computing providers, other than Salesforce.com, don’t consider integration. Perhaps they are thinking, “If you use our cloud, then there is no reason to sync your data back to your enterprise. After all, we’re the final destination for your enterprise data, right?” Wrong.

Cloud computing is a great way to go in many instances, but if you think you don’t have to provide core data integration from the cloud computing platforms to the on-premise systems, you have another thing coming.

Indeed, there are many reasons to provide data integration within cloud computing problem domains. The first and foremost is that you need to maintain an up-to-date copy of your enterprise data on-premise in case of trouble that could come in the form of cloud computing outages, clouds going out of business, or clouds purchased by companies that have no interest in staying in that business. There are many examples of these occurrences now, and they will only get worse in the future.

However, the primary purpose of data integration within the context of cloud computing is to assist in driving processes between systems, on-premise and cloud-delivered, and to manage data integration across those very different and geographically dispersed platforms. Those who only had to deal with systems talking intra-data center have some new challenges when they consider cloud computing.

These are the problem domains that require, dare I say it, data integration architecture and strategic technology. This means that you need to consider all of the source and target schemas, and how you’re going to securely and reliably move data between those points, accounting for the differences on the fly. Moreover, you need to consider MDM issues, as mentioned above, as well, as security, and data governance.

Here are a few words of advice:

First, consider the overall requirements of the business. Sounds obvious, but many who deploy cloud computing systems do not have a complete understanding of the overall business requirements.

Second, focus on the holistic architecture, on-premise and cloud-delivered, including how they will and should exchange data to support the core business.

Finally, select the right data integration technology for the job, and do so only after taking everything into account. You’ll find that there are both on-premise and on-demand options, and in many instances you may have to mix and match solutions.

You can’t do the cloud without data integration, and until we get data integration right, you can’t do the cloud. Pretty simple, if you think about it.

While the market is showing signs of recovery from the "Great Recession" most state budgets have been feeling the squeeze from the lag in recovery. In a recent article titled The Sorry State of Finances, Liam Denning explained that, "55% of state revenue, before federal transfers, comes from personal and corporate income tax." Denning also stated that, "the first three quarters of 2009 were the worst for state tax since at least 1963."

There is an apparent lag between recovery in the private sector and a state receiving tax revenue. So what can states do about this problem while they suffer in the red? Mr. Denning said, "Since states can't run general funding deficits, closing gaps mean raising taxes, cutting services and resorting to one-time measures." Mr. Denning's list of solutions is certainly accurate, but does it include all options that states have? What about employing new technology to discover fraud or recover uncollected revenue?

The many disparate systems, data silos, and various records throughout any single Department of Revenue can result in missed revenue opportunities. Using technology to integrate these various systems and records to obtain a single view of tax payer across these systems is a first step in recovering lost revenue—refilling those coffers!

In a recent webinar Government Technology presented how new technology can be used for improving data quality to reduce fraud and increase revenue.

Are there other ways states can use technology to refill their coffers?

This week, I attended a CIO conference in Philadelphia, for the first time in a couple of years. It was really good to hear them talk about their experiences, goals and issues, and to talk with some of them one on one. It spurred a bunch of new thoughts, which I’ll be writing about in a series of upcoming blogs. This first posting is on CIOs playing offense.

In my last posting, I talked about the Dr. Jekyll/Mr. Hyde nature of data—data being both an asset and a liability. The CIOs talked about playing offense vs. defense. They really wanted to be engaged with business transformation and growth initiatives. But many were saddled with cumbersome, expensive IT environments that sucked up 90% of their IT spend and resources just to “keep the lights on”. In other words, they were spending almost all their time and energy playing defense.

A key reason why CIOs get stuck in defense mode is because data has become a liability. Their information and integration infrastructures are crumbling under the weight of the explosion in data, the rapid acceleration of business demands and the massive complexity and brittleness of the “integration hairball” present in the vast majority of enterprises. As chief information officers, information is their key responsibility, and they’re unable to get all the relevant data to the business when it’s needed.

To play offense, to drive the business instead of merely supporting it, CIOs need to start chipping away at the hairball, incrementally implementing a far more rational approach to data integration and information management. They need to make it a priority for the IT organization. They need to put the right people in place who have this as their sole focus. They need to make it clear, from the top down, that the status quo for tackling integration and data isn’t good enough any more. In short, if CIOs don’t want to continue to be back on their heels in a defensive posture, data integration needs to become a CIO priority.

Next posting I’ll talk about why data is transcendent over applications.

Over the last few blogs, we have stepped through a project to implement Information Lifecycle Management (ILM) on corporate databases.  First, we evaluated the target databases, then we determined the Business Objects and assigned retention periods to the data – including both Legal and Operational requirements.  Now that we are ready to start applying the retention policies and deleting data, it is a good idea to set up an archive database as an intermediate repository for business objects classified as legal records.

Rather than just deleting data out of the source, you should create an archive database that resides on lower cost infrastructure.  Create the archive table definitions identical to source and test the SQL statements defined in the previous step (classification).  A database archiving tool like Informatica’s Data Archive integrates this step into the product making it much easier to setup and manage.  Once data becomes read-only and is less frequently accessed, it should be moved to the archive until the retention period expires.  A convenient view for the source application to see the moved data should then be created. Using this view, business users have all the access they need to the aged data.

By moving a bulk of the information out of the production database, the application’s performance will improve drastically.  Additionally, all copies made for test, development and backup will be that much smaller reducing infrastructure needs and cost.

With the aged data residing in an archive database, you can then execute the data destruction process and start purging data in a central environment.  If you can consolidate all of your database records to a single archive, searching for records during an eDiscovery is easier because all of the structured content resides in one place. For operational data that has shorter retention periods, this information should be cleaned out of the source database on a regular, frequent schedule.

Before you can claim victory, there are a few cleanup steps that need to occur before you can realize the true benefit of deleting database data.  In our next and final blog of this series, we will review the final necessary tasks and then calculate our benefit.

Julie Lockner, President & Founder, CentricInfo

I am very fortunate to have a CIO role that extends beyond the traditional responsibilities of IT. Part of my role includes the strategy and implementation of the recently launched Informatica Marketplace.

The Informatica Marketplace has generated a lot of buzz with our customers and partners. Last week while I was presenting at the Pacific Crest Annual Cloud Computing Conference I heard the excitement about our new offering from many of the attendees. Mostly, people appreciated our continued thought leadership and commitment to providing an open platform to host solutions for data integration, data quality and data management. And yes, it is an open platform to host solutions that not only support Informatica, but other vendors too - even competitors. Our fundamental belief is that the hand of free enterprise will ultimately win, so we are willing to provide an open platform to do so.

Here are a few questions I've heard and answered over the last few weeks:

What is the purpose of the Marketplace? Our interest is to serve the needs of the broader data integration ecosystem. Informatica itself has over 52,000 active members in our TechNet developer network today. These members asked us to provide a platform for them to build and interact. Given our 15-year heritage, use cases have now been repeated and solutions have been built. Now, sellers can share these solutions with interested buyers. For buyers, they will enjoy faster time to market with high quality solutions evaluated and certified by Informatica. For sellers, they will have an open channel to reach buyers. Our goal is to promote the Marketplace community where proceeds will be reinvested into funding it.

Why is Informatica launching a Marketplace now? As mentioned, we have over 52,000 active members in our developer network. This community has existed for more than 10 years. Within this community, developers share downloads, but have no means to monetize their solutions, nor were they reviewed by anyone. The Informatica Marketplace provides the mechanism to monetize a solution. The bottom line is that our customers have asked for this capability and existing, successful Marketplaces (like salesforce.com's AppExchange) validate their success.

Can anyone contribute to the Marketplace and what can be contributed? Anyone with solutions around data integration, data quality and data management can participate. As described previously, this is an open market. We envision solutions will be vertically or platform (horizontally) focused. They may be data models, mappings, mapplets, tools/utilities, methodologies, white papers, connectors, etc. We believe the wisdom of crowds will prevail and market demand will dictate which solutions will be offered. Our initial approach is to work with our User Groups to determine the solutions they need and want, prioritize the responses and recruit developers to fulfill these needs. The "How to Contribute" web page is going live in mid-March and we will begin posting the most popular solutions on the site so developers can select the ones they want to support.

Will assets be validated and what do you envision assets to sell for? The Informatica brand is known for its quality. In keeping with our brand image, all assets on the Informatica Marketplace will be evaluated in order to receive a "seal of approval" or certification. This certification will have different criteria depending on the type of solution contributed. We expect solutions will range in cost from $0 to the tens of thousands of dollars. Regardless, all products will be evaluated for quality and value before becoming available.

How many solution do you expect at launch? Our goal is to launch with 25 high demand, high quality solutions. We have had the opportunity to speak with leaders of other successful marketplaces and the consistent feedback is to focus on quality over volume at launch.

Does this replace TechNet? The TechNet download capabilities will move to the Informatica Marketplace. However, TechNet still provides tremendous value by enabling the developer community to collaborate and the core purpose of TechNet will remain in place to provide an inclusive, independent and interactive forum for sharing data integration knowledge, showcasing solutions, impacting technology trends and developing cutting-edge skills.

Will Informatica employees contribute? Yes, we are creating a group called Informatica Labs for our employees to contribute solutions. These solutions will be free and all contributions will be identified with a "labs" logo.

There are many more questions to be answered. My commitment is to answer them as quickly as possible. We realize that we must be fluid in our ability to change the Marketplace dynamically to meet the demands of our buyers and sellers.

Do you have any questions for me?

Today Informatica announced that Bay & Bay Transportation has used Informatica Cloud Services to achieve a six month project payback on their total Salesforce.com investment and a 900 percent return on their critical data integration project.

Bay & Bay is using the turnkey, on-demand Informatica Cloud Services to provide robust, bi-directional synchronization between its logistics management databases and applications with Salesforce CRM. Using an intuitive web based integration wizard, the company automatically maps source and target fields, configures powerful data transformations, and sets automated synchronization schedules. And being a true multi-tenant cloud-based service, there is no hardware, software, or infrastructure for them to install, manage, or maintain.

According to Rob Adams, CIO at Bay and Bay Transportation:

“If Bay & Bay didn't have Informatica, we couldn't have done any customer relationship management. By taking charge of our sales operations and putting all the sales information in one place, the Bay & Bay sales teams are devoting less time to administration and spending more time closing deals. Our sales managers can also use the real-time visibility into their team’s activities to forecast sales more effectively. Overall, if Bay & Bay achieves a one percent increase in sales revenues from this Informatica implementation, it will translate into a 900 percent return on investment.”

Earlier this year Chris Boorman predicted that, “Cloud Data Integration will eclipse both security and availability to be the defining capability that drives Cloud Computing success.” Informatica was also recognized as the top-ranked data integration application on the AppExchange in 2009. Ron Papas, Informatica Cloud general manager, explains the impact of Informatica Cloud Services this way:

“Informatica Cloud Services helps growing companies, such as Bay & Bay, quickly and cost-effectively address today’s data management challenges regardless of their business systems landscape, while retaining control and doing so in a secure manner.”

Check out our cloud customer success stories to learn more.

This is the last of the Data as an Asset series and what better way to wrap up the theme than with a view to the future. As stated by Thomas Redman, author of Data Driven, “Your company's data is a key business asset, and you need to manage it aggressively and professionally.” The future vision then is around Agile Data-Driven Enterprises.

Data is a critical business asset. It used to be that physical assets—equipment, land, inventory—determined the value of an organization. Now, as we have moved into the “information age”, most organizations have come to realize that their data and information are also critical assets. But few have set themselves up to make sure they are managing and utilizing their data in an optimal way to really impact how they run their business. This includes the people, processes and technology needed to know where the data resides, to understand it, to clean it and keep it clean, to get it to where it is needed, when and how it is needed.

What is the value of data to an organization? More specifically, what is the value of effectively managing and utilizing that data? Armed with data that is timely and data that they can trust, organizations can rapidly uncover new markets, attract and retain valuable customers, eliminate costly operational errors and delays, deliver products faster and make smarter business decisions. In short, they can consistently outperform their competitors. These enterprises are data-driven.

Data-driven enterprises maximize the business value of their data by establishing the organization, processes and infrastructure necessary to manage their data as a strategic asset, ensuring that relevant, trusted data can be delivered quickly when, where and how needed to support the changing needs of the business. These firms will be referred to as Agile Data-Driven Enterprises.

To accomplish this vision, organizations will adopt formal management disciplines such as Lean Integration and data valuation accounting. At the present time, these practices appear to be leading edge – or even bleeding edge. In a few years they will be commonplace. To quote Erwin Drahl from Informatica, “Lean Integration is a discovery not an invention. That is why everyone will eventually be doing it.”

For the record, here is the complete list of postings on this series:

1. Should Data be on your Balance Sheet?
2. Valuing Data Using Managerial Accounting Practices
3. Calculating EVA For Data Assets
4. A Market-based Approach To Valuing Data
5. Case Study In Managing Data Assets
6. Data as a Liability

Two of the more common questions that arise when trying to effectively deploy Data Governance are; "Where do I begin?" and "What business areas should I include?".  If you start too narrowly, the value and credibility of the effort is questioned.   Be too aggressive, and delivery risk and scalability become a problem.  As usual, success comes down to defining and managing scope.   However, more times than not it is prudent to err on the small side, and here's why…

Most organizations are more comfortable making smaller decisions, the likelihood of success is greater, and small failures are less costly (both in capital and in reputation).  Besides, if you start small and are successful, you can always grow.   But if you go big and fail, your odds of a second chance are diminished.  So how do you keep the scale down, but still deliver something meaningful?   Follow the money, because it's all about value.   The quote-to-cash and procure-to-pay lifecycles are rife with opportunities.   Go and speak with business and operational leaders.  Familiarize yourself with your "customers'" strategies and objectives.   Get your hands on the IT project portfolio.   These channels are excellent sources of information about what problems the business is trying to solve.   Once you've captured a handful of opportunities, do some basic analysis and fact gathering.  Then short list it to the two or three you think provide the most value and the best likelihood of delivery success.  Here's your starting point.

Good luck!

2009 was a banner year for Informatica's OEM business. We grew the number of ISVs (Software Companies), BPOs (Business Process Outsourcers) and SaaS/Cloud providers who use the Informatica platform as part of/with their offerings by 24 new partners. What was really interesting throughout 2009 was the variety of use cases across the spectrum of new partners addressed by various platform capabilities. You can read about this breadth and more details on specific examples in "2009 OEM Momentum".

For software, application or service providers faced with data acquisition, data conversion, data quality, data exchange, or other related challenges in the cloud or on premise, using the Informatica platform can alleviate that pain and allow them to focus on their core business.

With the delivery of Informatica 9 and the acquisition of Siperian, I expect that variety and growth to continue in 2010!

A key benefit of implementing an Application Information Lifecycle Management (ILM) project is to reduce the amount of structured data in the data center.  Application ILM is a combination of a strategy and process that assesses information based on its business value and aligns the technology it resides on.  This process assures that the data center does not over allocate IT resources if the business doesn’t need it.  And likewise, if the business can provide detailed requirements for what it needs for its data, the IT department has a better idea of its technology forecasting needs.  Application ILM is a capacity planner’s friend.

The first task in assessing database data for ILM is measuring the how much storage is consumed by the databases targeted for ILM.  Collect from the database administrators (DBAs) and the storage team what each believes they are consuming in storage.  The DBAs should provide a list of active databases they are using, used and allocated storage and growth rates for each.  The storage team should provide a list of databases they see from the storage allocated and storage allocation growth rates.  Many times there are databases and allocated storage out there that the DBAs weren’t even aware of.

The comprehensive list of databases should also include all production copies such as test and development copies, any copies used for standby, disaster recovery or backup.  Make sure to account for copies of data in a data warehouse or reporting system.  Additionally, from an IT perspective, make sure to identify the type of storage the copy is residing on.  For example, production copies are usually on Tier one infrastructure while backups may be on Tier two disk or tape.  Each Tier of infrastructure should have its own acquisition cost per gigabyte.  Once you have the quantity allocated and price by tier, you should be able to come up with a model for cost of ownership.

Once the current and projected storage allocation for the database data has been determined, now you can begin the task of classifying data based on its business value.  In the next blog, we will take the list of tables in the primary database copy and classify information in it based on business processes and data objects, legal and operational usage, and IT infrastructure.

Julie Lockner, President & Founder, CentricInfo

The only way for HealthDetail to hold or increase margins was to think differently. The company’s provider directory business supplies accurate and compliant healthcare directories for Medicaid and other health insurers. The problem was that the volume of data coming in and the breadth of different sources it was derived from threatened to overwhelm the existing database. HealthDetail could either invest in dedicated database administration resources, at great expense, or consider a more innovative solution: cloud-based data integration.

By standardizing on Informatica Cloud Services, HealthDetail is integrating its cloud-based Salesforce CRM data with MySQL databases quickly and easily. There’s no requirement for dedicated IT resources, and the team is almost immediately productive accessing, cleansing, and integrating the provider data. Using this cloud computing model, HealthDetail has reduced the time needed to process and submit directories for healthcare providers from 14 days to 5 (a 64 percent reduction), thereby enhancing customer satisfaction and optimizing efficiency. Moreover, the company has eliminated the requirement for contract database administrator staff, and the cloud model means there’s no dedicated IT hardware or software to support. The Informatica Cloud has also supported revenue growth in HealthDetail’s market research line of business.

Concerns over operating margins have vanished into the clouds…

• Read the press release
• Read the complete case study
• Watch the video

One aspect of an Information Lifecycle Management (ILM) project that often gets overlooked is deleting data. Once information has reached the end of its usefulness, delete it. It is the single-most cost effective task you can execute on an ILM project. If you don’t have the data, you don’t have to store it, manage it, or worry about it getting into the wrong hands. Delete it.

On an engagement I am working on, we are about to delete multi terabytes of database data. The initial business case has identified more than $2Million in hard-dollar savings in the first three years in the initial phase of the project. We calculated the savings based on the cost avoidance of storage acquisition and the amount of resources (both people and data center utilities) required to manage that data. When measuring the ILM project impact as a whole, we have tallied more than $50Million of projected savings based on changing people’s behavior on how information is managed over its lifetime. Information can be a company’s biggest asset, but if not appreciated for how volumes of information can impact a company, it can become its biggest cost.

There are a few steps you need to take before you can pull the data deletion trigger. Over the next few weeks, I will walk through a recent engagement on how we evaluated database data, categorized information based on Legal, Operational, and Business classifications, applied Records Retention Schedules, looked at every single copy of data, and then finally we are deleting data. If you are about to embark on a data deletion crusade in your company, please share your comments, questions, experiences. We look forward to hearing how you archive and purge!

Julie Lockner, President & Founder, CentricInfo

Continuing with the Data as Asset series, in this posting I explore the negative side and what can happen when data becomes a liability. Physical assets such as buildings, equipment, or even money, can become a liability if not managed properly; buildings can become unsafe to work in, machinery can be dangerous to operate, and business investments can turn into money-sinks. Similarly, data and information systems can also be assets that provide economic value to the enterprise or they can be liabilities that destroy value or put the business at risk if not managed well. Here are three common scenarios.

Too much data is a liability. For example, data that is in your production systems but not being used can cost a lot of money and act as a money-sink. According to Wikipedia, disk storage costs have seen a reduction of 36 thousand times since 1989. The problem is that the volume of data being stored has just about kept up with the cost savings. Most organizations these days measure their disk farms in terabytes and increasingly in petabytes. Even with storage costs around $2,000 per TB these days, this is still serious money (note: a 1TB drive for your PC may cost about $200, but a high performance RAID configuration for your mainframe will be about 10x).

The storage cost of your production environment is just the tip of the iceberg. All the unused data also gets replicated to one or more test environments and to the data warehouse. It also needs to be backed up which requires extra backup tapes, labor and storage. Furthermore, it impacts business by slowing down end-of-period processing and real-time transactions. Finally, all that extra data increases compliance costs, data quality analysis and resolution times. When you add it all up, it doesn’t take long before the costs of unused data is in the range of millions of dollars per year. Check out some of Informatica’s Data Archive Resources if you’re not convinced.

Wrong data is a liability. As the many stories at www.doyoutrustyourdata.com show, data can be liability as well as an asset. Examples range from sending life insurance renewal notices to dead people, misplacing a decimal point and selling products at a fraction of their cost, or simply writing off tens of thousands of dollars because the sales system doesn’t reconcile with the accounting system.

Data you can’t find is a liability. A good case in point is the data management woes at Commander Communications in Australia. As stated in this news report, "…a large number of orders were simply lost. The company’s computer systems were in such a mess that it failed to lodge its accounts on time and was suspended by the stock exchange, in the last straw for the stock price.” It was soon after this report that Commander went out of business largely due to their challenges in integrating systems and managing data. While not all data management disasters have such dramatic outcomes, the instance of failures is not uncommon. Furthermore, compliance and legal cases are common in many industries For example, medical products companies, financial organizations, and insurance companies to name a few are often challenged to defend themselves against decisions made 10 years ago or more. If you can’t find the data quickly when you need it, it can cost millions in fines or damages.

To bring this back to the theme of the blog series, data is an asset if managed properly with adequate governance, skilled resources and appropriate tools. Don’t let data become a liability for your organization.

Come back next week for the final posting in the data asset series in which I look to the future.  In the meantime, please keep those comments coming.

Business modernization programs typically focus on process standardization to gain the benefits of efficient repeatable, measurable processes. Enterprise resource planning (ERP) technologies fulfill the process standardization requirements and have now become a central point for management of business processes. However, ERP systems do not prevent low quality data from entering the systems nor do they measure its impact on the efficiency of a business process. Most organizations today are using the same ERP systems (SAP or Oracle) that were configured by the same consultancies. Therefore, the uniqueness and the scope for competitive advantage of any organization are defined by the people and the data.

Master data management (MDM) may be part of the longer term solution; however, MDM will not deliver short-term value. The challenge is in the time paradox where the priority of the business is to run operations and address the critical business issues to deliver immediate value, while an MDM project, in contrast, is a strategic initiative focused on delivering longer-term value.

How can we combine the near and far?

Only an economically focused, non-invasive and progressive approach, taking the short and long term business objectives into consideration, will be accepted and executed by the business.

Only those organizations with an effective data strategy embedded within the business strategy will be able to turn data into a real competitive advantage, deliver short and longer term value and ensure business success and sustainability.

Dr. Walid el Abed is the founder and CEO of Global Data Excellence.

As the market continues to consolidate and companies sell off assets, not only are the physical assets separated and sold, so are the digital assets – or liabilities. John Schmidt covered it in one of his recent blogs.

When the information to be separated and sold resides in a database, you need to understand the data model and determine what master data, or reference data, and transactions belong to the new company.  In the case of separating master data this may involve understanding the relationships between multiple systems to make sure that when the data is moved into the new company, it maintains context, is accurate and complete.  With transactional data, you need to know the tables and rows that comprise a complete transaction so that when data is moved, no orphaned rows are left behind.

When the data model is simple, this process can be fairly straight forward.  If the system is complex, such as Oracle E-Business Suite or SAP, using a data separation product like Informatica’s Data Subset can speed the process and eliminate any risk of breaking master and transactional data relational integrity during the process.

In one project, we did the analysis and comparison of splitting a company’s data using a team of consultants versus using a packaged solution.  It was impractical and nearly impossible to meet the deadline without having prebuilt templates for the master and transactional data available for use.  Informatica’s Data Subset solution provides predefined templates for packaged applications that reduces the amount of time needed to set up the data separation tasks.  Through the use of drop down boxes and wizards, the user can select the business organization and other criteria that would be used to select and separate the data.  Additionally, the routines are prepackaged to move the data into a new system while deleting the data from the old.  Logs and reports are available for auditing purposes to prove which data was moved, deleted and when. To do this job without an off-the-shelf solution, the time and cost would have been more than 300%.

We will continue to see mergers and acquisitions shape the market. Software solutions that are purpose built to create data subsets can significantly speed up and simplify the task of splitting those companies’ biggest asset – its data.

Julie Lockner, President & Founder, CentricInfo

I have been traveling to different cities on the Informatica 9 World Tour, talking to customers and partners about their painpoints around data and their ability to support the needs of the business.  In these discussions, one topic that has resonated strongly is the Dr. Jekyll/Mr. Hyde nature of data.

We all know that there's simply more information and data out there nowadays.  More types of data, coming from more places: partners, customers, suppliers, cloud applications.  Exponentially growing volumes.  More people, both business and IT, getting involved with the data.  Decisions and processes happening more and more in real time.

In many ways, this is a very good thing.  This means you have more information to drive smarter decisions.  To drive business operations with more data.  You can really tighten relationships with your extended ecosystem by sharing data effectively.  You can respond in a more intelligent, more automated fashion to events as they arise.  But only if you are able to effectively integrate and manage all the data.

The flip side of this coin is that these very same trends can make data a complete nightmare to deal with.  If you have a complex integration hairball, you probably don't have the flexibility in your architecture to deal with these new types and sources of data .   You may not have the infrastructure in place to address all the data quality issues that will abound.  Your business users, instead of being armed by all this data, will become overwhelmed by it.  Your costs will go up, and your quality and responsiveness will go down.

We have been talking a lot about the data driven enterprise.  One way to define a data driven enterprise is an organization that is able to ensure that data remains a valuable asset to the business, rather than becoming a major liability.   When we've asked for a show of hands at the World Tour, the majority of folks in the room indicated that their data is both an asset and a liability right now.  It's clear that your data can really be a double-edged sword if you don't have the methodology and technical foundation in place to manage it.

And often, even though we all say that data is one of our most important assets, data is really treated as though it were a liability.  Typically, the primary responsibility for integrating and managing data lies with the IT organization.  And most IT organizations are treated as cost centers.  John Schmidt has had some good postings on valuing data as an asset.  Would love to hear from you on what it will take to shift the mentality from data as a liability to data as an asset.

Las Vegas, as we all know, is the city of conventions, fantastic shows, great restaurants, and of course casinos. But one thing struck me on this visit that I hadn’t really noticed before. Each of these aspects of Las Vegas are world class in and of themselves. But when you really start to think about it, they are even more attractive as a complete set of activities which the conventioneer can pick and choose from to maximize the value of the trip – me, I chose to spend the day at TDWI sessions, dine at Bouchon, catch the Beatle’s Love show, and twice placed $5 down on #9 at the roulette table. Each of these is world class by themselves, but together, they provided an integrated experience that was greater than the sum of the parts.

This package of education and entertainment mirrors what is happening in the data integration world – providing a unified architecture for the frequently required capabilities across multiple data integration project types. But magic happens when the components are world class in and of themselves.

With Informatica 9 we have done just that. At the core is PowerCenter which is a clear leader in Gartner’s Magic Quadrant for Data Integration. Adding to that is Informatica Data Quality, also a leader in Gartner’s Magic Quadrant for Data Quality, and Informatica MDM (formerly Siperian) is the number one leader in Forrester’s MDM Wave. The list of innovative technologies goes on including Information Life Cycle Management, Complex Event Processing, Identity Management, and others. Each of these is world class and stands out on their own.

Your challenge, select a data integration platform that provides the best in class components — one that can satisfy multiple project types and scale with your needs. Putting your money down on Informatica 9 is the sure bet.

BTW, in case you are interested, both times I played #9, I won. Seems that 9 is a lucky number.