Clinical Data Repository

Whether you call it a clinical data repository (CDR) or a clinical data warehouse (CDW); it seems everyone (oh, all right - at least a lot of companies) are either building, or planning to build one. Most of these companies have grand notions of bringing together data from a wide variety of sources to be able to glean new insights. However the big unanswered questions are; just how are these insights to be gleaned, and who will be doing the searching to glean these insights?

Many advocates will tell you that a variety of software tools will be provided that can generate cool graphics. That these tools will be given to the end users, such as medical experts, in a self-service model so that they can peruse the data to their heart’s content. Whereupon doing so, they may find all sorts of interesting new insights. Does this sound a bit too utopian to you? It does to me. In my experience there are several things wrong with this view.

Data repositories are built using traditional relational databases. The average end user is not going to know how the tables are organized, not to mention what types of joins are required. Nor should they. You may well answer, 'well that’s what data marts are for - to provide a customized view that meets the needs of the end user.' The problem with this, is that it presumes that you know what the end user will be interested in looking at and how. It is a bit of a contradiction to presume that we can know how a user will want to explore the data such that we can create predefined data marts; yet at the same time espouse that the end users need unfettered access to explore and discover new insights.

Not only does the traditional data mart presume that we know what the end user will be looking for, it also is heavily biased in favor of structured data. If we are seeking to discover new insights, it stands to reason that these insights will likely be fueled by new data and ways of seeing the data. This means that we need to be able to support a variety of unstructured data in the data repositories, in addition to the traditional structured data. More importantly, we need to be able to query against both. To support such queries that will provide meaningful results across a variety of data sources, we must acknowledge that these sources may refer to the same concept with different terminology.

In reference to the self-service model, I trust you agree that the end users should not be expected to understand the organization of the data base in order to be able to effectively query it. Likewise, they should not be required to learn a specific software package – very few end users have the time or inclination to learn one. In addition, the use of web search engines have set expectations that the user should be able to simply enter their question and generate a response.

The challenge is, how do we address these concerns? There is no one single answer. One option is to simply abandon the concept of the self-service model and provide a team of data base and visualization experts to perform queries on behalf of the end users. While this may work in the short term, it will stifle the unfettered exploration, hinder the ability to find new insights, and diminish the value proposition of the data repository. The other more complex and ambitious option is to leverage a variety of tools and techniques to address these challenges. This would include use of NOSQL (Not Only SQL) databases, natural language processing, ontologies, and dictionaries to build a repository concept that more closely matches real world expectations.

Clinical Data Repository

Seems Everyone Is Building Clinical Data Repositories, Now How Are We Going To Use Them?

FIND CONTENT BY TYPE

Pharma IQ COMMUNITY

ADVERTISE WITH US

JOIN THE Pharma IQ COMMUNITY