Unlock Your Data’s Potential with Revelate
Revelate provides a suite of capabilities for data sharing and data commercialization for our customers to fully realize the value of their data. Harness the power of your data today!
Sections
intro to
Data exchange
Data Exchanges as a Mutually-Beneficial Business Opportunity: Getting Started, Tools, & More
Though many things about the modern world are changing at a rapid rate, most business metrics that indicate success haven’t changed. Every business wants to grow its revenue, keep its customers happy, reduce operating costs without sacrificing quality, and improve its processes.
One of the ways that businesses have been achieving these goals is by interacting with their data. Analyzing and using internal data to make data-driven decisions has been a norm in the business world for decades. But internal data doesn’t tell the entire story surrounding a business. With data being the new force for driving innovation and improvement, relying solely on internal data only gives a narrow view of how a business stands in an interconnected world. This is why data sharing has recently become a big draw for organizations.
The idea of sharing data isn’t new. There’s evidence to suggest that academic researchers, as well as the scientific community, were the first professionals to engage in data sharing. Today, more and more professionals and entire industries are sharing their data to meet their goals—whether it’s to generate more revenue for a business, improve operations and processes, get the information they need to develop software, programs, and applications, develop or improve products and services, or better understand customers. In pursuit of these initiatives, data sharing networks, or data exchanges, have emerged.
But today, data sharing is more of a business requirement than an option. Gartner predicts that by 2023, organizations that promote data sharing will outperform their peers on most business value metrics compared to those that do not. However, in the same vein, less than 5% of data sharing programs will correctly identify and locate trusted data sources.
Collaborative data sharing is a must is today’s modern business world. One of the ways organizations can partake in external data sharing is through data exchanges. In this article, we uncover what data exchanges are, explore their benefits, and get deeper into building a data management plan to support collaborative data sharing.
What is a Data Exchange?
A data exchange happens when data is seamlessly shared between organizations and systems—from the source schema to the target schema—without being altered or changed.
Data exchange is accomplished through a data marketplace (this doesn’t inherently indicate that data has been sold—data marketplaces can facilitate sharing as well as selling) containing a select group of organizations or individuals, which the data marketplace owner controls via invite-only or by offering more accessible membership via certain criteria. Data products are made available through publishing, complete with metadata categorization and descriptions so that group members can discover them through search functionality. The purpose of data exchange is to simplify the data acquisition and integration process by creating a collaborative, frictionless experience for participants.
Best Practices for Data Exchanges
Anytime data is being transferred from one entity to another, best practices should be followed. For data exchanges, best practices can be disseminated down into two main factors:
1
Establishing trust: All members of the data exchange should be able to trust the data that they are providing and receiving. This should be the basis of the data exchange. Ensuring that data transfer takes place in a secure environment (i.e., with robust data governance, security, access, and licensing policies) and that security is consistently and effectively enforced for every transaction.
2
Data exchange approach: Taking a close look at dataset characteristics and the data environment (source and target) help establish an appropriate approach to exchanging data. This likely means a combination of robust automations and manual processes.
Types of Data Exchanges
There are a few different types of data exchange, depending on how the data exchange owner wants to facilitate the transfer of data:
- Private data exchanges: With a private data exchange platform, only specific participants are allowed to engage in data transfer. Participants are usually invited by the data exchange owner, and must meet certain criteria (e.g., be a business in a particular industry, have data that meets certain use case requirements)
- Public data exchanges: Platforms were exchanges of large amounts of data from various industry verticals and sectors are encouraged. AWS is an example of a large public data exchange service where certain data products are available to anyone (while still allowing private data exchanges within the ecosystem).
- Peer-to-peer data exchanges: This is a more direct exchange of data between two organizations, or two departments or business units within one organization.
Why are Data Exchanges Important?
Organizations of all sizes in every industry realize that the vast amounts of data available, from transactions, sensor data, geographical data, and much, much more, can be leveraged in several ways, from generating additional revenue streams to improving processes and business operations. There are limitless advantages available to organizations that augment their data with that of third parties, and data exchanges offer an easy entry point for organizations to harness the power of data.
Better Healthcare and Treatments Through Data Exchange
For instance, initiatives for the interoperability of healthcare systems have been a hot-button topic in the healthcare industry. As populations age and people live longer due to advances in medical science, data sharing and interoperability of healthcare systems are essential. Facilitating these systems to connect and “talk” with each other is critical for ensuring that all healthcare providers involved in a patient’s case have the most current, correct information to deliver better care.
Further, the secure sharing of electronic health records (EHRs) helps doctors gain the information they need to make better, more informed diagnoses. At the same time, medical researchers can benefit from this information to create and improve medications and treatments.
The Office of the National Coordinator for Health IT in the United States was tasked with developing a trusted data exchange framework to democratize access to health information across all 50 states. This is called the Trusted Exchange Framework and Common Agreement (TEFCA), which went live in 2022. The TEFCA model aims to provide a single, scalable entry-point for interoperability participation, complete with robust privacy and security policies, required and optional exchange purposes and patterns, and incorporation of the Health Level Seven International Fast Healthcare Interoperability Resources standard into the roadmap. This ensures that patient data is protected without eliminating the insights that doctors and researchers can glean from it.
Internal Data Exchange for Better Operations
We’ve relied on concrete as a solid and durable building material for thousands of years. And even today, the way that concrete is made hasn’t really changed: it’s still a mix of roughly 10% cement, 20% water, 30% sand, and 40% gravel. But of course, the precise way that mixes are created is what sets a high-quality concrete mix apart from a low-quality one. Further, continued material shortages are adding to the challenge of concrete producers to optimize what they use to the absolute letter.
Optimizing your mix design, raw material utilization, and delivery processes are paramount for success in the ready-mix concrete industry. When an organization has many different plants operating in silos, monitoring the efficacy at every level of the concrete development and delivery process is difficult—potentially leading to one plant functioning at a higher level than others. But through interoperability fueled by automation, data exchange between all systems, plants, and machinery is possible to address the above challenges and lead to better, more efficient operations.
Taking advantage of the ability to share data with other key players can no longer be ignored; it must be embraced for a business to be successful in the modern world.
Main Benefits of Data Exchanges
While data sharing has traditionally been a difficult process, data exchanges have simplified how companies buy, sell, and share data. A combination of effective data management, including storage and cataloging, as well as governance, is handled partly by the data exchange platform and partly by the data provider, helping to ensure that the process of buying, selling, or sharing data is streamlined while following the proper licensing and security protocol.
Further benefits of a data exchange include:
1. Reducing Time to Market
The time it takes to break into new markets or discover untapped opportunities within existing ones can be significantly lessened by augmenting internal data with third-party data sets. From a strategic business perspective, gathering as much information as possible to support data-driven decision-making makes sense.
2. Breaking Down Data Silos
Through a data exchange ecosystem, data silos that naturally build up within departments, partners, and business units can be broken down due to the ease of data transfer. Through a customizable Data Web Store like Revelate, organization-wide access to data—as well as access for external partners—is democratized without compromising on security.
Specific views can be provided that separate internal and external access, ensuring stakeholders only have access to specific data.
3. Enabling Secure Data Sharing
Traditional data sharing methods like emailing spreadsheets or files, conducting batch processes with FTP or ETL, and using APIs isn’t secure or scalable. They only end up wasting time, money, and resources. Further, they copy data, which can affect its integrity, and the result is stale, static data that quickly lives out its usefulness.
A system like Revelate allows access to data from its original source, either as-is or prepared and packaged according to specific options listed on the Data Web Store. The entire process, internal or external, can be automated to reduce the burden on IT teams. Revelate allows either self-service data discovery and retrieval or the same process facilitated by a salesperson, administrator, or another employee.
Unlock Your Data’s Potential with Revelate
Revelate provides a suite of capabilities for data sharing and data commercialization for our customers to fully realize the value of their data. Harness the power of your data today!
4. Controlling and Governing Access to Your Data
Exchanging data doesn’t mean losing control of it. Data exchange platforms allow you to control and govern access to your data regardless of whom you’re sharing it with. Data governance policies and procedures that your organization has already established can be adhered to with a platform like Revelate. Automations can be established so security and access policies, as well as data licensing, can be applied automatically. If manual processes are preferred at any stage, that’s an option as well.
5. Allowing a Business to Better Leverage Data for Improvements
There are two main ways that businesses can leverage data as a result of a data exchange:
1
Democratizing access to data within the organization, avoiding data silos, and delivering data to those who need access.
2
Using third-party data to augment internal data, filling gaps and giving better, more detailed insights.
In both use cases, data exchanges allow businesses to take full advantage of data to apply to every aspect of the business. This could be developing or improving products and services, identifying and eliminating bottlenecks and barriers that exist in different processes and procedures, improving customer service and the customer experience, and much more.
Data Exchanges Vs. Data Marketplace
Generally, the terms data exchange and data marketplace are used interchangeably. The majority of marketplace platforms can handle data transfer in various capacities, including between private groups or public members, or a combination of both. But there is a distinct difference between the terms data exchange and data marketplace, and it’s important to understand this difference, even if one platform can facilitate both of them.
| Data Exchange | Data Marketplace |
| A place to provide data to others for no financial gain (e.g., providing base code to help with development of a specific process for a platform, such as Cisco engineers providing code for Cisco users) | A place for buying and selling data |
| More focused on helping others glean business value from a platform or ecosystem through data exchange rather than monetary gain | More focused on gaining financial benefit from selling data products (for the data seller) or purchasing data that has a direct or indirect financial benefit (e.g, purchasing lab safety data to help develop safer and more efficienct processes for handling hazardous chemicals) |
| Data exchange is typically more based on networks and relationships | Typically more one-sided with a more linear sales process |
Getting Started with Data Exchange
Data exchanges break down siloed data in organizations and make the process of data discovery, leveraging data, and sharing or monetizing data more streamlined and effective. Data exchanges also allow organizations complete control over data management and consumption, including data governance.
In order for your organization to take full advantage of the benefits of data exchange, however, the preferred approach to exchanging data must be determined.
Determine How You Want to Exchange Data
| Data exchange strategy | Approach |
| Internal data exchange: | Sharing data across globally-distributed business units, departments, and subsidiaries while maintaining data integrity and security has traditionally been difficult. The intra-company data exchange strategy focuses on providing a centralized location for approved and secure data that can be shared across the organization, facilitating better collaboration and better data-driven insights. |
| External data exchange | A data discovery or sourcing strategy needs to be established. Having a centralized portal is one method where data providers can publish their data is essential so that your data scientists can analyze it and determine what can and cannot be used. |
| Data distribution | By using a data marketplace solution your organization can avoid the cold-start dilemma when attempting to reach a target audience of data consumers. A data marketplace like Revelate allows you to personalize the experience, and reduce data transfer costs by eliminating reliance on outdated methods like APIs or FTP. |
Your organization doesn’t have to choose one strategy and stick with it forever. Hybrid strategies can be developed that suit different purposes and business goals. Given that online data exchange platforms uncover new business opportunities, it’s natural for an organization to adopt hybrid data exchange strategies over time.
Data Management and Governance
Two essential components of participating in a data exchange are data management and governance. While both are crucial parts of data science and are related to each other, they describe different aspects of handling data.
Data Governance vs. Data Management
| Data Governance | Data Management |
|
|
In essentially every way, data management and data governance work together create a framework for handling data within an organization. While data governance handles the security, access, accountability, and integrity of data, data management is the technical side that handles data storage, categorization, and movement.
Data Governance Model
The first step in creating a data governance strategy, is to create a data governance model. A data governance model outlines an organization’s data privacy and security processes and systems and applies to every aspect of data handling throughout the organization. This includes creating, storing, maintaining, and disposing of data.
The data governance model includes the following essential components or “pillars”:
1
Transparency. Trust goes hand-in-hand with transparency, and trust is critical for data governance to be successful. Employees and business stakeholders (including leadership) should be involved from the beginning and be willing to gain an understanding of how data is collected and stored and how it moves through the organization. By involving everyone from the beginning, trust is established and easier to maintain as the organization’s data governance policies evolve.
2
Accountability. Those who are most familiar with a particular type of data should be the ones responsible for it. For instance, financial data should be handled by the finance department manager in an organization, while the HR manager would handle employee data. An effective data governance model takes accountability for data beyond IT and gives it to every relevant stakeholder, including managers, supervisors, and more. These individuals become responsible for the data in their respective departments.
3
Integrity. Ensuring that organizational data is high-quality is paramount. Without high-quality data, data governance can’t be effective. This means using technologies like data cataloging and data governance tools, and securing data in the cloud, mainly since organizations typically use multiple cloud-based systems (e.g., an organization may use the Google Suite in combination with the Microsoft Suite, which both provides cloud data storage functionality). Further, relevant stakeholders in an organization should have an active role in data-related discussions, and honesty and transparency (e.g., people should feel safe letting someone know when a data-related mistake or breach happens) should be prioritized.
4
Collaboration. Effective data security needs a collaborative focus behind it. This means eliminating data silos and ensuring that anyone who interacts with organizational data, including users, understands their responsibilities to ensure that data governance and security are maintained.
From there, a data governance strategy needs to be developed. At a high level, implementing a data governance strategy includes:
- Establishing the scope of data governance and objectives. This requires a careful look at how organizational data is already dealt with, including data management and how data moves through the organization. Data architecture, storage, security, and data quality should all be considered.
- Data stewardship programs should be defined and implementedData stewardship programs should be defined and implemented. Specifics should be outlined regarding how data should be handled at each stage of the data lifecycle process, which includes understanding where organizational data originates, defining data quality thresholds, identifying critical data (data that is essential to the organization’s success or needs to be retained for regulatory purposes), creating and defining operational procedures (data creation, collection, and storage), ensuring that communications standards are in place (between systems and programs), and creating metadata management standards.
- Data standardization and quality goals. An organization should determine the most important areas of data focus, which is the data that impacts business decisions most. This could include data from regulatory reports, data relating to a specific business metric, or data that improves business processes, products, or services. Further, the risks associated with poor-quality data should be identified, and the data with the highest risk should be prioritized to fix or take action on first.
The heart of data governance is collaboration, transparency, honesty, and trust, all surrounding the need to control data quality, enforce business rules, and ensure the resources for effective data security and management are available. This requires all stakeholders throughout the organization to be on the same page.
Managing the Governance of Combined Data
While typical data exchanges make bringing data together easy, there are still the essential steps of enterprise data governance, integration, and interoperability to consider. Organizations must take great care to address these challenges before they can gain optimal business value from the data they’re getting from data exchanges.
Data obtained from a data exchange should be carefully assessed to ensure it’s correct and appropriate.
Suppose documentation isn’t provided or doesn’t provide the full context surrounding the dataset, such as where the data originally came from, how it was initially used, its consistency in time, its relationship to other data, and other metrics and rules. In that case, the data should be used with caution.
Master Data Management
Master data is an organization’s non-transactional data that can be used as a single source of truth. The specific characteristics of master data are outlined in the table below:
| Master Data Characteristic | Explanation |
| Less volatile | Master data doesn’t frequently change like other data might. For instance, a customer’s name, phone number, and address would be considered master data. This data is often static between multiple systems when the appropriate integrations are in place. |
| Complex | Master data can become quite complex—for instance, information for numerous customers for a parent organization with multiple child companies or subsidiaries. To handle this data correctly, multiple processes must be in place to ensure that data accuracy is maintained. |
| Essential to decision-making | Because master data doesn’t often change and is considered a “golden record” or “single source of truth” data, it’s often used in the decision-making process as it’s the most trustworthy. |
| Non-transactional | Master data is non-transactional, meaning it isn’t used for applications and systems that handle transaction processing. |
Data Management in the Cloud
Enterprise organizations may use multiple cloud-based systems, from document storage and collaboration (e.g., Google docs) to general data storage (e.g., cloud storage on AWS).
Managing data in multiple cloud-based systems presents a layer of perceived complexity. Since different cloud platforms have different policies and procedures when it comes to data governance, data management in the cloud may seem difficult or even unachievable for organizations without having to take some sort of risk: either potentially getting fined because of changing laws and regulations or not being able to keep up with them, or slow data operations down to a crawl to ensure compliance.
But centralized data governance and security while providing democratized access to data is possible, ensuring compliance with the latest laws and regulations, even as they change. The answer to the riddle is data automation. Data automation automatically makes data governance and security happen, based on specific triggers. This ensures consistency and allows organizations to implement new policies to the automation path as needed.
Revelate, for instance, is partnered with Immuta, a leading data security organization, to ensure that your organization’s data governance policies are followed throughout the data fulfillment process.
Immuta works by centralizing data governance while at the same time being platform agnostic so that organizational data governance policies can be applied to all cloud-based platforms that the organization uses. At the same time, Revelate is platform agnostic regarding data fulfillment, meaning that data can be extracted from and distributed to any platform. With the two technologies working seamlessly together, data automation becomes central for an organization’s effective data management in the cloud.
Unlock Your Data’s Potential with Revelate
Revelate provides a suite of capabilities for data sharing and data commercialization for our customers to fully realize the value of their data. Harness the power of your data today!
Using Data Discovery Tools
Data discovery refers to the process of an organization “discovering” its data, whenever it may reside, and using that data to obtain insights and make more informed business decisions. Data discovery tools are essential for this process, especially for discovering sensitive data that may be “hidden” from view due to its confidentiality, combined with data governance and security measures.
Sensitive data includes:
- Personally identifiable information (PII)
- Payment information (PCI)
- Protected health information (ePHI)
- Any data that is protected against unwarranted disclosure
Controlling and managing sensitive data, as you can imagine, is essential for organizations. Data discovery and classification tools can help find sensitive data, determine its ownership, and organize in such a way that ensures its protection. Data governance initiatives and security can be built around sensitive data, and help organizations understand the associated risks with it and how to mitigate those risks in the best possible ways.
For instance, the partnership mentioned above with Immuta and Revelate is an example of mitigating risk as data is moved and accessed in an organization. Through automatic data governance, including security and access protocols, anyone who wants to access a specific dataset will be subject to automated checks and balances. These processes determine if the person has authorized access to the dataset as a whole, or only to specific parts of it. According to their access, the requested data will be provided to them.
Building a Data Management Strategy
Another essential part of successful data exchange is to create a data management strategy or plan. Regarding data and being in control of its movement and access within an organization, it’s better to be proactive rather than reactive in its management.
A data management plan comprises the processes, procedures, and tools of an organization’s data usage roadmap. It provides holistic guidance for working with data at every point in the organization’s data science pipeline, from collection to fulfillment to deletion, alongside data governance initiatives and policies.
Building an effective data management plan requires several considerations, including:
Understanding Business Objectives
As your organization produces and gathers data, it’s essential to have established business goals in terms of what to do with that data, including how it can be used to benefit the business directly.
Otherwise, there’s a risk that your organization is wasting valuable time, money, and resources analyzing data that isn’t useful. Data that isn’t useful to the organization should be deleted or archived according to established data governance protocol.
Setting business objectives using data requires planning and thought. For instance, if the aim is to use data to generate revenue, then you’ll want to look at how the collected and stored data contributes to data monetization initiatives.
Monetizing data can be internal or external:
- Internal data monetization refers to using data to improve business operations, including processes, procedures, and policies, to save money and resources through increased efficiency and productivity. It can also include using data to develop new products and services or improve existing ones.
- External data monetization means selling the data to data consumers, typically using a data marketplace. The data marketplace usually handles data transfer, and either the data provider or the data marketplace takes preparing and packaging the data product following the data consumer’s specifications, as well as licensing and access considerations.
Understanding business objectives is just one of the considerations that are involved in building a data model. A data model is a framework that asks what data should be captured and how it should be used in different contexts to meet business goals. Data modeling also considers privacy and access privileges with regard to sensitive data and helps maintain the integrity of data as it is shared or sold.
Having Strong Processes for Collecting, Preparing, Storing, and Distributing Data
The processes that an organization has in place for data collection, preparation, storage, and distribution are essential considerations for a successful data management strategy.
Identifying the appropriate data stakeholders related to these data processes means asking the following questions:
Data Collection
- What will your data sources be? How can data be collected from those sources?
- Most organizations will need to consider the collection of structured, unstructured, and semi-structured data—are the appropriate frameworks in place to support these varied types of data?
- Will data extraction (from various sources) be done manually or automatically via pre-determined processes?
Data Preparation
- How will data be cleaned and transformed to prepare it to be analyzed adequately?
- How will incomplete or disparate data be identified and handled (e.g., a service such as Dun and Bradstreet could be used to fill in missing data points from various industries and sectors)
- Are the appropriate data cataloging (i.e., naming, tagging, categorization) conventions in place to sort data, as well as lineage, and metadata assignment, in place to support data discoverability?
Data Storage
- Where will data be stored?
- How will data be stored (e.g., lakehouse, data lake, warehouse, etc.)
- How will data be kept secure while it’s being stored, and while it’s in movement?
Data distribution and analysis
- How will data ownership and stewardship be determined (e.g., by department, by potential use-case, etc.)
- How can data access be democratized while still ensuring security and access privileges are adhered to?
- How will insights gleaned from data be communicated to a non-technical audience (i.e., executive staff)
Strong Data Governance
Another essential part of building an effective data management process is strong data governance. Ensuring that access is controlled at every stage of the data pipeline while still ensuring that democratized data access is prioritized throughout the organization is essential to support the effective use of data and an organizational culture that supports data sharing.
One of the ways that an organization can approach data governance is by using the SMART (specific, measurable, actionable, realistic, and timely) principle. This involves developing information principles, which are universally understood rules for information management that business stakeholders, data professionals, and partner organizations can apply and follow.
These principles should be relatively static and act as a gap filler between other policies and procedures surrounding governance and business objective fulfillment.
Other considerations when developing SMART framework data governance rules include:
1
Using desired business outcomes to heavily influence rule creation. This will likely result in more high-level rules being developed first, with more granular, specific ones following them.
2
Ensuring that a data-driven organizational culture is maintained. This involves engaging with business leaders and stakeholders to ensure that everyone speaks the same “data language” and understands how granular processes, procedures, and principles contribute to the overarching purpose: archiving positive business outcomes.
3
Deploying SMART principles throughout D&A governance processes. Different governance principles need to be applied depending on the business scenario. Adaptive D&A governance embraces consistent thinking across data and analytics governance but allows for autonomy and agility for teams to make certain decisions to create value for the business.
Prioritizing Data Literacy within the Organization
Implementing data-related initiatives in an organization is great, but if not everyone understands those initiatives, then the efficacy of certain processes, policies, and procedures can be lost on some individuals. For a truly democratized approach to data management, all stakeholders that may interact with data can benefit from training and knowledge sharing to increase data literacy.
Understanding Data Management Trends
Data management process isn’t static. As new technologies emerge that provide improved data augmentation and automation, and with the cloud being a significant contributor to these technologies, things are moving fast in the world of data management. Data is also increasingly distributed through multicloud, intercloud, and hybrid architectures, so understanding the emerging technologies available to help with management and governance is essential.
One of the frameworks that have been developed to help data professionals keep on top of changing trends in the field of data management, as well as data science in general, is the Hype Cycle. This yearly publication has been developed by Gartner, and is often shared by major players in the data industry such as Databricks.
In terms of pushing the needle forward in the world of data management, the innovations of particular note include:
1
Data management in the cloud: Essentially, every data management technology available today is developed in the cloud, meaning that organizations are using the cloud exclusively for data management. This makes data management more complex, as organizations may be dealing with multiple cloud-based systems with different governance, access, and data storage rules. Technologies such as intercloud data management (links multiple cloud deployments together in a holistic solution), augmented FinOps, private cloud bdPaaS, and data fabric (provides consistent capabilities across multi-cloud environments) are coming out of the woodwork to address the complexities brought forth by the cloud.
2
Augmented data management: How data is stored after it’s collected is one consideration, but as the data moves around, has more data added to it, and otherwise is changed should be considered in its effective management as well. DBMS and data integration, augmented data quality, cataloging, and the use of AI and ML are all factors that need to be considered.
3
Data management operations: The growth of cloud-based technologies alongside hybrid and edge computing is affecting operations management for data management. Effective data management and governance across multiple platforms are demanding on FinOps, DataOps, data engineering, and multiple other professional data-related sectors.
4
Data management systems for analytics: Emerging technologies that support simple, but effective management of large amounts of data are being developed, including data ecosystems, data fabric, and the concept of a lakehouse (combines elements of a data warehouse and lakehouse for one, holistic data storage solution). All of these technologies are changing how data is used in analytics, BI, AI, and ML.
Data Management for Different Industries
While there are standardized approaches to effective data management, different industries may have different needs for managing data, whether due to regulatory requirements and laws, or industry-defined best practices.
Clinical Data Management
Healthcare organizations, including medical research facilities, strongly benefit from having access to clinical data.
What is Clinical Data?
| Clinical data example | Description |
| Treatment history | Treatments received, including medications, physiotherapy, psychotherapy, data on patient outcomes with/without these therapies and medications |
| Diagnostics | Medical tests performed, results of those tests, research and other factors that led to diagnosis |
| Family relationships | Genetic and hereditary information that provides insight into a patient’s risk factors for certain diseases, illnesses, or conditions |
| Symptoms | Ailments that the patient has been experiencing that affect their quality of life or have led to a diagnosis and/or treatment |
| Ambulance and hospital records | Records of hospital stays and trips that the patient has taken in an ambulance, including location and any treatments/medications given during those events |
Sources of clinical data include medical records, patient surveys, and administrative databases.
Clinical data management specifically includes collecting, cleaning, and managing patient data that is collected due to participation in a clinical trial. Collected data must be stored and secured in accordance with laws and regulatory standards. HIPAA, for instance, is an American legislative Act that dictates how healthcare organizations must take responsibility for ensuring that patient data is protected, while at the same time recognizing that the flow of health-related information, including patient data, must not be stifled to ensure that high-quality healthcare is able to be provided and conducted based on the insights gleaned from healthcare-related data.
What this means for medical researchers and healthcare providers is that strict data governance initiatives need to be in place to ensure that patient data is protected while it’s being stored, as well as while it’s in transit. Part of this governance is ensuring that robust and granular access privileges are determined and adhered to by organizational data systems.
Let’s use Revelate as an example. As a data marketplace solution, Revelate is able to incorporate centralized and robust data governance to support the demands of clinical data management. Available datasets are displayed via a fully customizable online interface, ensuring that data consumers can view and understand data products in the best way possible. When a data consumer wants to access a dataset for purchase or otherwise, automated data governance protocols perform the necessary checks and balances to ensure whether the data consumer is able to access the entire data set, a portion of it, or a modified version of it. Marketplace display options also control who can view available data sets based on their level of access.
Unlock Your Data’s Potential with Revelate
Revelate provides a suite of capabilities for data sharing and data commercialization for our customers to fully realize the value of their data. Harness the power of your data today!
Legal Data Management
Law firms must follow strict data-related laws and regulations like the healthcare industry. While there are no specific overarching federal laws regarding data storage or protection requirements, there are a variety of granular laws and regulations imposed by separate jurisdictions and states. For the legal industry specifically, American Bar Associations typically suggest that law firms take “reasonable precautions” to ensure the security of their client’s data, including ensuring that the cloud provider (since they will likely be storing client data on the cloud) doesn’t have access to the data. Connecticut’s State Bar Association addressed this specifically in an informal newsletter back in 2013. One of the important points that this newsletter mentions is that client data must be segregated from lawyer data to prevent unauthorized access.
In Canada, the way that law firms are expected to store client data is governed by overarching federal legislation (PIPEDA) but also by each province’s law society. For instance, in British Columbia, if the law firm uses cloud services to store client data but the cloud service indicates that the data will be stored somewhere outside of Canada, then the law firm has the obligation to inform the client and receive their consent to store their data in this way.
Obviously, effective data management is central to a law firm’s operations. Not only do they have to contend with client data storage, but they also have to consider how they’ll store other forms of important data, including legal knowledge management. Chances are, a law firm will need to use multiple cloud-based systems to meet its needs. This means that data management will need to be done on multiple platforms, all with different interfaces and approaches to security and access.
Unifying data management, in this case, is important. When a law firm can manage the security and access part of its data from a centralized location, it makes it easier to ensure that the firm remains compliant.
Customer Service Data Management
Customer data management describes acquiring, organizing, and using data from an organization’s customer base to drive decisions and initiatives that lead to better customer service outcomes and, ultimately, happier, more satisfied, and loyal long-term customers.
Like clinical data, customer service data often includes sensitive information that organizations must keep confidential. This includes customer names, phone numbers, addresses, and more sensitive details like credit card numbers, social insurance or security numbers, driver’s license information, payment information like bank account numbers, and more.
One part of effective customer service data management is ensuring that an organization’s data is accurate and of high quality. Impacts of poor-quality data can cost organizations millions of dollars due to inaccurate business decisions resulting in financial loss.
Retail Data Management
For retail organizations, data is the key to developing better products, better customer service, managing inventory effectively, and establishing thorough yet streamlined processes with suppliers, distributors, and brick-and-mortar stores.
Data can be used to extract insights from across the value chain, from how customers browse the retailer’s website, how they interact with social media channels, and in-store purchases. This is important because customers increasingly expect personalized shopping experiences, and often get frustrated when these experiences are not delivered to them at all, or correctly.
Consider the following examples of how retailers typically deliver personalized shopping experiences:
- Sending a customer an SMS with information about sales or promotions when they are in the vicinity of a brick-and-mortar location
- Upsell recommendations on complementary or related products when the customer goes to their cart on the retailer’s website
- Personalized product recommendations based on previous purchases sent via email
To be able to execute these initiatives, retail organizations need to be able to gather, prepare, store, and analyze data effectively and efficiently. If data analysis that leads to the delivery of these personalized experiences fails, then it significantly affects the success of business outcomes.
Data Exchange Platforms
There are various data exchange platforms available, with more and more entering the mainstream market every year. Popular options are discussed below, describing what makes them stand out as common data exchange platforms.
AWS Data Exchange
Built by Amazon, AWS is a cloud-based data exchange platform that has thousands of data products available for organizations to access in one place. Key advantages of using AWS data exchange include:
- Wide variety of high-quality data. With over 3,500 data products from over 300 data providers, finding specific, high-quality data for various use cases is easy with AWS data exchange. There are also over 1,000 free data products for organizations to utilize.
- Ease of data discovery. With cloud-native service for various data types, including data files, tables, and APIs, finding datasets that match your preferences and format is seamless.
- Centralized procurement and governance. Subscriptions from third-party providers can be migrated to AWS at no additional cost, allowing you to manage all subscriptions in one place. Governance is also simplified; the platform offers one place to exchange data publicly or privately depending on your needs, as well as simplified contracts and consolidated, secure billing.
- Modern data technology. Data in AWS can be integrated natively. The system also encrypts data that is in transit and at rest. Integration with AWS Identity & Access Management (IAM) (a web service that allows you to control access to AWS resources securely) is also included.
1
For data providers, AWS data exchange makes data delivery, entitlement, and billing easy by taking care of these processes natively instead of needing the data provider to build or handle these processes themselves. Data providers benefit from a secure, transparent, and reliable platform where they can list products that can reach AWS’s wide customer base.
2
For data consumers, the main draw of AWS data exchange for data consumers is its ease of access to a wide range of data products. With a subscription, you can access thousands of products from data providers. In addition, you can use the AWS data exchange console as a centralized platform to manage, create, view, and access data sets for use across other AWS services, including analytics and machine learning. This functionality is included with an AWS account, so if you already have an account, you have access to the features mentioned above automatically.
Snowflake Data Exchange
Snowflake is a well-known data fulfillment platform with options for buying, selling, and sharing datasets. As Snowflake has thousands of users from various industries, the existing data repository is attractive for new and current members. Aside from the user base benefit, Snowflake data exchange offers additional benefits:
- Customizable ecosystem. Snowflake allows data providers to create a branded internal or private ecosystem version of a data marketplace with full control over what data products are listed and who has access to them.
- Centralized data access. Business analysts and data science teams can browse and access data products in a secure and curated environment.
- Invitation access only. Users can invite others to become data providers and data consumers on their Snowflake data exchange ecosystem.
- Data sharing and monetization. Provides the ability to monetize data and share it on one platform.
- Augment internal data. Snowflake provides the ability to augment existing internal datasets through bi-directional data interchange with third parties.
1
For data providers, Snowflake data exchange allows full, centralized control over data governance for internal and external sharing and monetization. Providers can manage membership statuses, grand and revoke access to data via listing options, audit data usage, and apply granular security controls to datasets.
2
For data consumers, secure access to data is provided, enabling increased collaboration with departments, business units, customers, and other users in the Snowflake ecosystem. Better, more accessible access to data means that more educated decisions can be made and provides increased insight into how other areas of the business, as well as outside the business, operate, helping to eliminate silos and encourage collaboration.
Cisco DevNet Ecosystem Exchange
Cisco is a large, global network of businesses and organizations (Cisco partners) that build solutions. The Cisco DevNet Ecosystem Exchange is a platform that connects those partners by allowing them to provide data that can lead to new business opportunities, solutions, and technologies being developed. The platform is comprised of two exchanges: Code Exchange and Ecosystem Exchange. With code exchange, developers and independent software vendors (ISVs) can collaborate through learning and sharing code. Ecosystem Exchange provides a centralized location where data discovery of built solutions can occur.
The main features of the Cisco DevNet Ecosystem Exchange are:
- Access to over 1500 business solutions. The Ecosystem Exchange is a rapidly growing platform that provides buyers and sellers a wide range of Cisco solutions.
- Promotes collaboration and sharing. By having a centralized platform where like-minded individuals can easily access and provide solutions built for Cisco, natural connections and networks form, encouraging more collaboration within a business or entire industry.
- Amplify data products to a willing audience. Data consumers on the Ecosystem Exchange are actively looking for Cisco solutions that data providers offer.
The Ecosystem Exchange is a great place for data providers to amplify and market fully ready Cisco-built solutions.
For data consumers, the Exchange allows easy access to useful data products that can be used to build new or augment existing solutions.
The main features of Code Exchange include:
- Easy access to Cisco-compatible code. This provides access to code created and maintained by a network of over 500,000 developers within the DevNet community, including Cisco engineering teams.
- Buyers can leverage code to jumpstart development. Cisco product teams, developers, and ISVs provide sample code and applications to demonstrate how to use product APIs and build on top of Cisco platforms and solutions. This allows businesses to get up ad running with their solutions faster and easier.
- Easier discovery of relevant code. Filters are available to help users identify code examples by technology, programming language, or recently updated and recommended. Code examples can be run in the browser-based integrated development environment for fewer local dependencies.
1
For data providers, Code Exchange allows them to get their code in front of talented developers, who can test it and use it for different purposes, allowing the data provider to glean insights from their work that can be used to influence future projects and business decisions.
2
For data consumers, Code Exchange solves one of the major challenges developers face: finding the exact code they need to solve a problem, jumpstart a project, etc. The easily searchable database with granular filters helps developers zero in on the code they need rather than spending hours sifting through pages of results.
Challenges of Data Exchanges and How to Overcome Them
Because data marketplaces and data exchanges are often the same platform, the challenges of data marketplaces also apply to data exchanges. Even though exchanges are not often focused on financial transactions, the interface is largely the same as a marketplace: users are still searching for datasets the same way whether they are purchasing them or simply downloading them.
For most organizations, data monetization or data exchange isn’t their core business, so they use a data marketplace to sell or share their data. But using marketplaces as a single marketing and exchange source comes with a problem. Data marketplaces are controlled by a single operator, who is usually a cloud provider. This limits a company’s ability to display custom products and test different listings, which can hinder discoverability, especially when a data exchange has many data providers. Getting the right technology in place to distribute monetized data makes data discovery, marketing, and dissemination significantly smoother.
1. Seller Differentiation
Typical data marketplaces opt for standardization over customization to keep their UI intact. Metadata is used, but the styling of all data listings looks the same or very similar, not allowing for much differentiation for data providers. Further, these platforms don’t work like search engines, so similar data product listings may get obfuscated by other sellers depending on when you list them.
Some marketplaces or exchanges attempt to mitigate this issue by having search filters and other granular search options, but the fact remains that newer postings typically get prioritized before older ones, especially on a general landing page.
Solution: A fully customizable data marketplace solution.
Your Revelate data marketplace is fully customizable to reflect the experience you want, with self-service options for external customers and differentiated internal access options for employees and partners.
2. Technical Expertise
Navigating packaging and productizing your organization’s data sets can be difficult without the proper technical expertise. To get the full benefit from using data, the challenges of integration, interoperability, context-specific governance, and categorization must be addressed by the organization itself since a typical data exchange doesn’t do it. This is compounded by the fact that data exchanges facilitate bringing data from multiple sources together, adding to the effort an organization needs to manage it all effectively.
Relevate’s data automations handle moving datasets from various sources, preparing them, packaging them in specific configurations based on customer needs, and placing them on your data web store. This eliminates any requirement for technical expertise to fulfill data orders, taking the burden off IT and allowing anyone, such as salespeople or other internal employees, to fulfill data orders quickly and easily.
Solution: Automations
3. Data Discovery
Many data exchange platforms don’t offer the ability to filter and customize data sets, making it difficult for customers to find exactly what they’re looking for in a sea of data products. Further, your organization may have a specific way of classifying datasets that don’t align with the limitations of a typical data exchange platform.
Solution: Dataset categorization
Creating a fully customized web store through Relevate allows you to incorporate the categorization aspects your company uses for datasets, resulting in a tailored experience that fits your business objectives.
Using Revelate as a Data Exchange Platform
As a data fulfillment platform, Revelate provides a holistic data marketplace solution—one that can facilitate data exchange, monetization, and sharing effectively, both internally and externally. While many platforms can provide this functionality, Revelate makes it accessible to organizations and businesses of all sizes and levels of data maturity through automation.
The entire data fulfillment process, from extraction to delivery, can be automated through Revelate. Security, access, governance, data preparation and transformation, and finally distribution can be handled fully by automated processes. In short, Revelate solves complex data fulfillment problems by providing simple, user-friendly solutions. Instead of focusing on the granular aspects of data fulfillment, data scientists can instead focus their efforts on other, more pressing tasks. At the same time, data access is democratized through Revelate by enabling those who require access to data to have it, while still following organizational data governance and access policies.
When data is accessed, we ensure that the data consumer or system has the right permissions to do so. This may mean that they have access to an entire dataset, a portion of it, or no access to it at all. Data is always encrypted in transit and the data provider controls how data is encrypted at rest. We help ensure that data remains secure at every step of the distribution process.
Consumers can easily search for the data they need via a fully customizable marketplace, meaning that the marketplace owner can control how data products are displayed according to what works best for the organizations participating in the exchange. Once data is purchased or downloaded, the data consumer inherits the licensing and entitlement metadata.
Conclusion
In a modern business environment, data is essential. No business can thrive today without using data effectively, whether to improve business processes, operations, products, and services or package internal data insights and sell them as a data product to gain a reliable and consistent revenue.
With data being a sought-after resource, different platforms have emerged that facilitate data transfer between organizations and individuals. Among these platforms are data exchanges. As organizations in different industries increasingly understand the value of sharing their data for mutual benefit between provider and recipient, exchanges are the perfect avenue for facilitating this transfer of data. A data exchange can be public (accessible to anyone with or without an account), private (only accessible to certain members of a group), peer-to-peer (only between two organizations or business departments), or hybrid (some parts of the exchange are public, while other parts are private). Data fulfillment providers often facilitate the technical background processes behind these exchanges, supplying the interface, data process automation, and licensing applications that support effective data exchange.
Organizations that want to participate in data exchange still need to take the time to develop effective data management and governance for the data that their organization owns and uses. This process should involve every stakeholder, technical data expert, and data users.
Data exchanges also have their challenges, namely with limited product listing display options (based on UI) as well as a lack of relevant product prioritization (newer listings are typically prioritized to show on a landing page versus older listings, even when they may be more relevant). Data providers and consumers should be aware of these challenges when using data exchange platforms so they can work alongside them.
A comprehensive data fulfillment solution is within your reach, regardless of the size of your organization. Book a demo of Revelate today and discover the power of your data.
Unlock Your Data’s Potential with Revelate
Revelate provides a suite of capabilities for data sharing and data commercialization for our customers to fully realize the value of their data. Harness the power of your data today!
