Unlock Your Data’s Potential with Revelate
Revelate provides a suite of capabilities for data sharing and data commercialization for our customers to fully realize the value of their data. Harness the power of your data today!
Sections
Intro to Data Marketplace
Types, Use Cases, Challenges, & More
While the data vendors and publicly available datasets have been around since the beginning of the internet, it’s safe to say that the idea of buying, selling, and sharing data has become increasingly popular in the last 10 years or so, mainly due to the sheer amount of data that human beings produce.
Data is everywhere and is quickly becoming one of the most valuable resources for organizations to buy and sell. Organizations of all sizes produce astronomical amounts of data per day. Being able to harness the insights that all that data provides is a huge opportunity not only for the organization itself but also for other organizations inside and outside their industry. In reality, the potential of an organization’s data is endless.
With more and more organizations realizing the potential of their data and wanting to take full advantage of it, they are seeking ways to get their valuable datasets into the hands of the right audience. Further, organizations often want to augment their existing data with data from outside sources—making that data more valuable.
So two challenges emerge:
1
Getting data into the hands of the right people, which applies to organizations looking to monetize their data (ensuring security and access privilege checks are performed)
2
For organizations looking to augment their own data, ensuring that the data they’re purchasing is high-quality
Questions that are likely to emerge as an organization tries to figure out how they can prepare, package, and sell their datasets include:
- Who determines what datasets should be for sale?
- Who will be responsible for preparing and packaging datasets for sale?
- How will we get datasets to the right audience(s)?
- How can we distribute datasets efficiently and effectively (following organizational and regulatory data governance)?
- How we can distribute datasets without requiring additional resources from our IT team?
On the other hand, organizations looking to purchase data may ask:
- How can I find the data that I’m looking for?
- How do we know that the data is relevant to our use case?
- How do we know that the data is high quality?
- Which formats is the data available in?
- How can I get the data into my data ecosystem?
- How can I determine if there are privacy and security concerns with this data?
- How does this data fit within our data governance policies?
A data marketplace is the answer to both scenarios—selling data or buying data. Marketplaces for data, on the surface, check a lot of the boxes for getting datasets to market that organizations need to address—getting data to a target audience, distributing data effectively—and are easy to understand (simply put data products online for sale). For purchasing data, similar checks would be performed for buying anything online, including reading reviews, liaising with others in your industry to see if they use a particular data marketplace, and even purchasing or requesting a sample for a few datasets to assess the quality and relevance.
In addition, most data marketplaces offer automation of data fulfillment—meaning that once a data request is sent, the system automatically performs security and access checks and prepares the data for distribution.
But data marketplaces, while effective for the most part, do present some challenges for organizations that want to monetize their data (data providers) and those who want to purchase data (data consumers). These challenges often relate to the limiting nature of marketplaces for data regarding how data products are found and displayed, convoluted buying journeys, and overall trust in the data marketplace.
But before we get into the specifics of data marketplaces and their challenges, let’s unpack what a data marketplace is, the different types, use cases, and more.
Your Revelate data marketplace experience is highly customizable in terms of data cataloging, segmenting, and marketing data products to suit your target audience(s). Revelate facilitates a seamless experience in terms of purchasing, selling, sharing, or exchanging data products, strengthening the relationship between data providers and customers.
What is a Data Marketplace?
The data marketplace faciliates data transactions by providing the following features:
| Data Marketplace Feature | What It Does |
| Data productization | Allows the creation and maintenance of data products for data consumers to purchase |
| Data discovery | Makes it easier to find relevant data products through a searchable catalog and metadata display |
| Data access | Access to data products is controlled through various mechanisms, such as APIs, access control integrations, and more |
| Data monetization | Data providers can license and sell their data products to data consumers |
| Data integrations | Data providers and consumers can send and recieve data from other sources and platforms |
| Communication (between providers and consumers) | Provides a platform to connect data providers and consumers, where questions can be asked, changes can be requested, and more |
For the data consumer, a marketplace for data is a medium (in the form of an online storefront) that makes it possible to find and purchase the datasets they seek.
From a provider’s perspective, a data marketplace is a platform that is used to list—and sometimes fulfill—their data products and offerings.
Traditional data discovery, integration, aggregation, and distribution methods may prove to be too time-consuming and complex for the average data provider, especially without a team of professionals and dedicated resources to handle the intricacies of licensing agreements and the custom development required for each data set. A marketplace for data provides a platform for data providers to not only sell their data with the marketplace handling complex licensing, security, and access but also use data marketplaces to augment their own datasets so they can become more valuable. This also allows the data consumer to become a data provider, which helps stock the data marketplace with a steady stream of data providers.
Data marketplaces can specialize in a specific type of data or only offer data from a particular industry. We’ll get into specific types of data marketplaces later in this article, but one example of this is an IoT (Internet of Things) data marketplace, which specializes in offering data from connected cars, smart homes, building devices, and more. Other data marketplaces, like Snowflake and Databricks, as example, offer a wide variety of data products and services that are not industry or use-case specific.
Data marketplaces can also facilitate sharing, access, and distribution of data with no monetary exchange, for instance, with an organization that wants to provide democratized access to data to stakeholders.
While the practice of buying and selling data products is relatively straightforward, the process does require certain levels of data sophistication and technical capabilities for every party involved in a data transaction.
- Data providers have to determine what data is worth productizing and selling and consider regulatory compliance, privacy and security, and any other potential concerns with selling a particular dataset, including data lineage and ownership considerations. In addition, available formats need to be determined, and pricing needs to be set.
- Data consumers need to be able to find the data they would like to purchase (this is a common challenge with data marketplaces, which is discussed later on in this article), be able to ensure that the data is high quality and will work for their use case, and be able to get the data into their ecosystem.
Data Marketplaces vs. Traditional Data Collection Methods
Manual data collection, as you can imagine, is not nearly as effective or accessible as purchasing relevant data products from data marketplaces. Manual data collection methods likeweb scraping are slow, resource-intensive, and expensive and bring about questions regarding ethics, security, and privacy regarding the collected data. Even big data collection, which is a more modern method of collecting data, is resource-intensive and requires solid data science initiatives like effective data pipelines, enterprise data governance, and more to execute properly.
Even one-off data deals between data consumers and providers tend to tip the scales in favor of the data provider, potentially giving them less incentive to provide high-quality datasets. On the other hand, data marketplaces have a higher incentive to provide high-quality data to retain customers and maintain their reputation.
Overall, it’s easier and more effective for organizations to use data marketplaces for data discovery versus conventional methods, and it’s easier for organizations that want to monetize their data to use a reputable data marketplace.
Key Opportunities Data Marketplaces Provide
1
Getting data to those who need it. A data marketplace solves one of the major challenges that data-driven organizations face, which is sharing their data effectively, either internally or externally. The data marketplace eliminates the difficulty that typically comes with fulfilling data orders, whether it be waiting for ticket assignment or managerial approval, or checking compliance documentation. Instead, data automation allows orders to be fulfilled via a self-service portal or through a salesperson or other stakeholder, and automates security, access, and compliance checks.
2
Building an ecosystem. A data marketplace allows a variety of data providers to gather to monetize data, which allows them to form an ecosystem where data is transferred and shared.
3
Finding new monetization opportunities. Data consumers (who can also be data providers) can use a data marketplace to create innovative new revenue streams for their organizations. For instance, a delivery company that uses a combination of GPS, real-time traffic data, weather, and local news on construction, police presence, and other information that may potentially cause delivery delays could provide that data to municipal transportation organizations so they can develop the most efficient and effective routes for bus systems. On the other hand, car dealerships can provide sales data to vehicle manufacturers so they can determine which vehicles are the most popular depending on a specific region (e.g., state, province, or country).
4
Finding new reselling opportunities. A marketplace for data provides a unique opportunity for data consumers to take advantage of the value of their data. Consumers who purchase data products from a marketplace can then use that data to augment their own, and the insights gleaned from that integration can be resold on the marketplace as a new data set. In this way, a data marketplace can encourage data consumers to become data providers, and encourage data sharing
5
Creating a data sharing business model. Data marketplaces don’t just have to be areas where buying and selling take place. They can also support data sharing and exchange. An organization can create a data sharing business model with the marketplace as a platform for ensuring ease of internal data access (data democratization), or faciliating data transactions that don’t have monetary exchange, but instead provide mutual benefits for the data provider and consumer.
6
Creating a centralized area for data discovery. By using a data marketplace to create a data sharing ecosystem, data consumers within that ecosystem can have a centralized location to discover new data products and sell their own.
7
Helping to ensure data quality. Data marketplaces can enact service and licensing agreements to ensure that the data products that are being sold are consistently high-quality and comply with security and privacy regulations.
Steps to Productizing Data
Creating data products effectively involves a multi-step process called data productization. A data marketplace uses pipelines and integrations to facilitate this process, so that safe and compliant products are available to be purchased or retrieved on the marketplace.
The steps for data productization are:
1
Manufacturing: Creating standard and custom data products for a general or specific user base.
2
Packaging: In this stage, the data products are made to be discoverable, marketable, and attractive to data consumers
3
Fulfillment: Data product purchases are handled either automatically or manually, regardless of where the data product originated
4
Distribution: The data product is made available for data consumers in whichever formulation they require (e.g., direct download, API-based data streams, etc.)
5
Commercialization: Different types of marketplaces, including public, private, or hybrid, are created, each able to handle the complexities surrounding licensing, payment processing, and service integrations
Your Revelate data marketplace experience is highly customizable in terms of data cataloging, segmenting, and marketing data products to suit your target audience(s). Revelate facilitates a seamless experience in terms of purchasing, selling, sharing, or exchanging data products, strengthening the relationship between data providers and customers.
Types of Data Marketplaces
There are a few different types of data marketplaces, and they each exist to control access levels for organizations and individuals and have different approaches to monetizing or sharing datasets.
Approaches to Data Monetization
Marketplaces have different approaches to how they provide data and the audiences they cater to. These approaches can be broken down as follows:
- Providing data for multiple use cases, industries, and audiences: To attract data providers from a wide range of industries and give them more opportunities to sell useful data products to consumers in different industries and who want to use the data for different purposes, data marketplaces can define specific meta formats and abstractions. Examples of marketplaces that take this approach are Snowflake and Databricks.
- Industry-specific data: some marketplaces provide data to specific industries only. Bloomberg and Informatics are two examples of data marketplaces that only provide data for the financial services industry.
- First-party data: Some data marketplaces provide data that they themselves generate, which is typically aggregated data (e.g., IoT or financial data from stock markets). A specific example of this would be Revelate’s own client, CME Group.
Access Control For Data Marketplaces
Each type of marketplace can be configured to be a public data marketplace (also known as an external data marketplace), private data marketplace (also known as an internal data marketplace), hybrid marketplace, or multi-layered marketplace. This allows customization of access and permissions depending on the intended audience.
Public Data Marketplace (External)
- Accessible by anyone at any time via a publicly available link
- Provides a wide range of datasets from one or more data providers
- Inherent trust is different from an internal marketplace, so the data provider and the marketplace provider (which may be the same or not) have an obligation to ensure that the data is safe to be shared and doesn’t violate appropriate regulatory compliance laws
- Users of external marketplaces should expect Service Level Agreements (SLAs), data access restrictions based on local governance and/or the purchase and usage of the data product
Private Data Marketplace (Internal)
- Only accessible by certain parties via a private link as determined by the marketplace owner
- Typically used by a single, data-rich organization (e.g., enterprise) to manage and share first and third-party data internally
- Enables streamlined, low-risk data access to employees, teams, business units, etc.
- Teams can catalog, classify, and make general and specialized data assets available while controlling user roles and permissions
Hybrid Data Marketplace
- Some parts of the data marketplace are available to view publicly, while other parts can only be accessed via a private link
- Combines elements of a public data marketplace and a private data marketplace to offer one platform where controlled data product access can take place for internal and external data consumers
- Controls licensing for different levels of user access, so for instance a large dataset could be fully available for internal use, while only a subsection of that dataset is available for external use
Multi-layered Data Marketplace
- Serves internal and external data consumers
- Provides regulated access to datasets depending on the “levels” or “layers” of the dataset, including raw, processed, or derived data that’s been aggregated, transformed, or enriched in some way so it can be offered externally
- Since the multi-layered data marketplace serves internal and external data consumers, it has the most complex data marketplace architecture due to the need for more security, access, and role-based permissions to be calculated with each transaction
It’s important to understand that regardless of the type of data marketplace, the goal should always be to employ data governance models that allow data from different sources, including personal, commercial, or government to be distributed while still respecting privacy and security rights.
With that in mind, different types of data marketplaces are as follows:
B2B data marketplace
Supporting data distribution and sales for organizations that want to provide their data to other businesses, a B2B data marketplace or business data marketplace offers an effective solution with a relatively low barrier to entry. Typically, B2B marketplaces that are successful make it easy for an organization to integrate their data into the marketplace or don’t require them to integrate it at all. From the consumer side, organizations often choose to purchase datasets and APIs from B2B data marketplaces because the data is typically analytics-grade, which means it’s better suited for immediate integration into a program or application for analytics purposes.
Personal data marketplace
While traditionally a marketplace for data was geared primarily towards businesses selling their data to other businesses, a personal data marketplace, also known as a public data marketplace or consumer data marketplace, has recently emerged as a way for regular consumers to monetize their own data.
With privacy and security being an increasingly important concern for the regular joe, regulations like theGDPR in the UK and the CPPA in Canada have emerged to provide rules to organizations in terms of how they handle consumer data. In an effort to put the power of their data back into the hands of the individual, a personal data marketplace allows individuals to sell their data directly to businesses on a secure platform that is compliant with the applicable regulations, depending on the marketplace’s location.
Most personal data marketplaces use blockchain technology to ensure consumer data security, authenticity, and credibility. Activity tracking and traceability allow consumers to gain visibility into who is buying their data and how it’s being used. Of course, like other data marketplaces, these consumer-focused personal data marketplaces don’t own the consumer data but are just a medium for facilitating data exchange.
IoT data marketplace
An IoT data marketplace is a platform where data providers can provide application data for data consumers to use for a variety of IoT devices, from consumer devices like smartphones and smartwatches to vehicle operating systems, manufacturing equipment, healthcare systems, and much more.
An IoT device constantly emits sensors and generates data, which companies can capitalize on to provide real-time digital signals from potentially millions of digital touchpoints. Harnessing and selling this data is a very attractive and lucrative option for organizations.
Open data marketplace
The idea behind an open data marketplace is that publicly available data is aggregated to make it easily accessible by anyone, including individuals and organizations. This can have a myriad of benefits, including improving services and products in both the public and private sectors.
One example of an open data marketplace is the United States’ own data.gov. This initiative was developed to make the government as a whole more open and accountable. When government data is accessible to citizens, academics, and private businesses, the idea is that the data can greatly assist in effective decision-making efforts, economic development, and countless other areas. The data provided on the data.gov open data marketplace is the result of collaboration between states, cities, and counties with the US and internationally to provide a robust data ecosystem.
With big data consumption becoming the norm for larger organizations, open data marketplaces make the most sense for gathering large amounts of data, especially for organizations that need to take advantage of large-scale data from various industries or sources like the ones listed above. In this way, an open data marketplace could be thought of as a big data marketplace, because it gives access to so many different datasets from a wide variety of businesses and organizations. Open-source software, like the Apache Hadoopecosystem, is a good, cost-effective choice for setting up the infrastructure and tools that your organization would need to manage large amounts of data.
Your Revelate data marketplace experience is highly customizable in terms of data cataloging, segmenting, and marketing data products to suit your target audience(s). Revelate facilitates a seamless experience in terms of purchasing, selling, sharing, or exchanging data products, strengthening the relationship between data providers and customers. Get Started
Types of Data Sold on Data Marketplaces
Any type of data set can be sold on a data marketplace, but the success of datasets usually depends on the usefulness that it provides in terms of enriching an organization’s existing data.
With that in mind, the most popular datasets that can be found on a typical data marketplace include:
| Type of Data Found on a Data Marketplace | Use Case |
| Business intelligence | Data used for business intelligence includes everything from statistics, business activities (operations, processes, workflows, etc.) performance metrics, reports, analytics, and more. Organizations use business intelligence data from external sources to augment their own, providing more in-depth insights that can be used to make better holistic and granular business decisions. |
| Marketing and advertising data (demographics, buyer preferences, etc.) | Marketing and advertising data (demographics, buyer preferences, etc.) A wide variety of marketing and advertising data is also commonly found in a data marketplace. This data helps organizations better understand their customers to market to them more effectively. |
| Firmographic | Like demographic data is information about the individual, firmographic data is information about organizations. Firmographic datasets can include information like organizational size, industry type, total sales and revenue, location, and more. One use case is using firmographic data to determine a business’s effectiveness in a target market compared to existing players. |
| Research data | Data from research studies, scientific endeavors, academic papers, and raw data from research labs and institutes are just a few examples of possible research datasets that could be found on a data marketplace. Medical labs and facilities, as well as research and academic institutions, can use this data to aid in creating new medicines and treatments or further their understanding of a particular subject matter. |
| Market data | In the finance industry, market data refers to data used for research, analysis, trading, and accounting for financial instruments of all asset classes on world markets. Market data comprises a wide variety of components, but the goal is to find trends, patterns, and other information to make more informed financial business decisions. |
| Industry data | As more of a broad term, industry data contains specific information about the economic activity of organizations in a particular industry. This helps research organizations and organizations that conduct similar business understand the bigger picture surrounding their industry, like labor information, average business size, financial values, and more. |
Why Organizations Use Data Marketplaces to Buy and Sell Data
Specific reasons why an organization would choose to use a data marketplace to buy and sell data rather than opting for another option, such as individual private sales or partnerships with buyers and sellers, include the following:
Revenue Opportunities (Data Monetization)
One of the main reasons an organization would use a data marketplace is to support their data monetization strategy. With a data marketplace, organizations don’t have to build and maintain the infrastructure required to sell their data on their own. Further, data governance policies are maintained by the data marketplace, meaning that the security and access that the organization requires for each of the datasets they are selling can be consistently applied. Immuta is one example of a data security platform that provides the tools for data and security teams to implement and execute data governance policies.
Gives Access to an Established Client Base, Which Avoids the Cold-Start Dilemma
There are a wide variety of data marketplaces, and many cater to specific types of industries or data types, making it easier for organizations to get their datasets in front of the right audience. This can also help with data product display, as for example, a financial data marketplace would understand what information financial data consumers would want displayed in metadata and support that with their product display functionality.
Trusted Marketplaces Give Organizations a Venue for Finding and Using High-Quality Data
A typical data marketplace provides a plethora of data with relatively easy access (you typically just have to create an account and go through a straightforward verification process) for an organization to use to augment their own. A marketplace for data is often used by vendors across the world, and many of the larger, more popular marketplaces have hundreds of data providers ready to sell their products. Typically, a data marketplace works similarly to any eCommerce website, where a user can browse through data products and use filters to find what they are looking for (although standardizations necessary for data marketplaces may make some datasets difficult to find—but we’ll discuss that in the next section of this article).
Many data marketplaces also allow sample data to be downloaded in advance, allowing a data consumer to try part of a dataset to ensure that it will actually work for their intended use case. Since data marketplaces have standardized ways that datasets can be displayed for consistency’s sake, it might not be entirely clear if the full data set will work. This makes requesting a sample necessary in some cases before committing to a purchase.
Further, a data marketplace worth its salt should have standards, meaning that data providers are vetted before they can place their products on the platform for sale. A data marketplace has a vested interest in ensuring the quality of data products that are made available on its platform, as consistently having low-quality data providers would affect the marketplace’s reputation.
Which Industries Use Data Marketplaces?
Because data marketplaces can be used to sell many different types of datasets, it’s no surprise that many different types of organizations take advantage of them to find the information they need to meet business and organizational goals.
Government
Open data marketplaces, which were described in detail earlier in this article as centralized platforms for citizens, businesses, and academic institutions, and more to share their data in an open format in an effort to improve the lives of people.
For government organizations specifically, these data marketplaces can provide various opportunities for improving overall operations. These opportunities include:
- Increasing public trust in the government. When government data is freely available to citizens and organizations to access and share, it’s easier to hold the government accountable. For instance, with Data.gov, the open data marketplace managed by the US government, the OPEN Government Data Act was enacted to ensure that federal agencies publish their information online as open data, using standardized machine-readable formats with their metadata included in the Data.gov catalog
- Creating a data-driven digital economy. As more and more data moves through our nations, it’s important that governments, from federal to state, provincial, or territorial, can harness this data to leverage the benefits of a digital economy. In Canada, for instance, the province of Ontario’s government is taking steps to improve its digital footprint to benefit all of Ontario’s citizens. This includes improving internet access and allowing online access to government services, but it also means leveraging provincial data from citizens and organizations—the same way that the federal government would leverage this data—to improve public services, strengthen the province’s economy, and provide more prosperous opportunities for everyone.
- Providing effective mass communication to the public. From natural disasters to health considerations like the recent pandemic, the wide variety of information out there can be difficult to validate. This makes it difficult for the average person to gain enough objective information to make effective, informed decisions for themselves. Going back to the pandemic as an example, open data from a reputable data marketplace (such as the aforementioned data.gov) in partnership with medical experts and scientists could be used to combat misinformation faster by providing accurate, effective mass communication of objective facts and easily-digestible information (via charts and graphs) to the public.
Healthcare
A healthcare data marketplace, especially open ones, provide countless opportunities in the healthcare industry that are aimed at improving the health of individuals. Some of these opportunities include:
Finding cures for rare diseases. According to the 2020 Global Data Access for Solving Rare Disease: A Health Economics Value Framework white paper by the World Economic Forum, approximately 10% or 475 million people are affected by a rare condition. Solutions (treatments and medications) to these rare diseases are available, at least according to Lynsey Chediak of the World Economic Forum, but the problem, as Lynsey is quoted as saying in the 2021 Data-driven Economics: Foundations for Our Common Future white paper, is that the data is siloed in isolated clinical records. Open health data marketplaces with secure data governance could provide answers to individuals living with rare conditions while still respecting the privacy and confidentiality of patient data.
- Data democratization in the healthcare ecosystem. The medical value chain begins when a person goes to see their doctor. A patient can interact with a wide range of medical professionals during the course of their lives, from nurses, doctors, surgeons, technicians, pharmacists, and more. Ensuring that patient data is accessible to all of these professionals is essential to ensure that continuity of care is maintained. A health data marketplace can help facilitate this by allowing individuals with a stake in the patient’s data—whether that be a healthcare professional or the patient themselves—to access needed data safely and securely.
- Creating more effective treatments. When information about patients, including diagnostic information and specific treatment plans for various ailments, is available on an open but secure platform, insights can be gleaned for better patient outcomes. The Mayo Clinic, for instance, created a large data marketplace with over 154 years of clinical data and deidentified it so physicians, researchers, and medical professionals can use it to improve diagnostics and treatments across the full spectrum of medical care.
Manufacturing
Manufacturing might be one of the oldest industries, with roots back to the industrial revolution in the 19th century, but that doesn’t mean that this industry can’t benefit from the power of data. Improving operations and making the creation of goods more efficient and cost-effective is paramount for the modern manufacturer. Here are the opportunities that data, and subsequently data marketplaces, can provide for this industry:
- Use of data analytics for improvements across the business. From predictive maintenance to optimizing manufacturing processes, connected factories (often referred to as industry 4.0 or manufacturing 4.0)—where all devices and elements can communicate with each other—gives complete visibility into high-level and granular operations. By analyzing the large amounts of data that these connected devices and elements create, transformative insights can be gleaned and applied to decision-making, optimizing how machines run and respond to different scenarios, such as errors or physical problems, and overall ensuring that product quality remains high while production is fully optimized. Over time, a connected factory where data analytics are used as an essential part of the decision-making process saves time and money for the factory as a whole.These manufacturing data analytics present another opportunity for the factory—packaging these insights to provide to another location or selling them via a data analytics marketplace to other factories in the same or similar industries to create an additional revenue stream. Connected factories can augment their own data to gain even more information—how do their manufacturing processes compare to others in the industry? Is there insightful information available from a partner or other third party that can enhance operations further? These are just a couple of examples, but the possibilities are limitless.
- Mitigating supply chain risks. Ensuring that supply chains are running smoothly is an increasingly difficult challenge. In Canada, leading factors that contributed to worsened supply chain challenges supply chain challenges in 2022 based on a survey of Canadian businesses included delivery delays, increased prices of products or supplies, supply shortages, and more. Addressing these challenges requires creative solutions. Predictive analytics programs can alert manufacturers about potential disruptions. Through a data marketplace, real-time information could be fed to a manufacturing facility, enabling these programs to be enhanced with real-time augmented data from partners and third-party sources, leading to more effective forecasting of disruptions that could impact the supply chain.
New product development. By gathering data from a variety of sources, manufacturers can gain a better understanding of their target markets and beyond, including consumer perceptions regarding their product and whether there is an opportunity for new product development based on consumer needs. By mimicking real-world conditions, the research and development required for new product development can be reduced, and speed to market can be increased while giving the product the best chance for success.
Finance
As an early adopter of data-sharing initiatives, the finance industry already understands the power of data. However, legacy technologies and approaches to finding and gathering data have proven to be challenging to maintain, especially with the astronomical amount of data this industry produces on a daily basis. A financial data marketplace provides the opportunity for financial institutions to utilize data effectively in the modern world through the following:
- Enhanced data discovery and delivery. Investment and trading firms traditionally source their data from direct exchanges and trading venues or from well-known data vendors such as Bloomberg and Refinitiv. But managing relationships with multiple data vendors and effective data discovery are ongoing challenges. What data marketplaces can provide is a robust cloud-based data ecosystem complete with the tools and infrastructure technology to enhance data discoverability (e.g., through better metadata display), better delivery options, and faster time to value (e.g., self-service, immediate downloads, or real-time data subscriptions).
- Eliminate the need to support aging infrastructure to consume data. Using traditional data vendors in the financial sector required the financial institution to work within legacy systems, including proprietary equipment, to visualize the data they received accurately. With modern financial data marketplaces, data can be accessed over the public cloud (without it ever leaving the host platform), enabling secure data delivery and also reducing operational complexity and costs with regard to data management for the recipient. With a data marketplace, you go to a website, click on a displayed dataset, and it immediately appears in your Snowflake, Tableau, or BigQuery environment, prepped and ready to use. With historical methods, a user would have to access a vendor’s FTP server, supply credentials, download a file, prep it, load it into a database, and so on.
- Better stock market predictions. Accurately utilizing the astronomical amounts of data that the finance industry produces on a daily basis effectively leads to the ability to make better investment decisions. Machine learning and Artificial Intelligence, when fed data, assist in making highly-educated predictions that can be applied to stock market trading, eliminating bias and influences and making trading data-driven and fact-based. This helps portfolio investors maximize their returns.
Automotive
The global automotive industry big data market was valued at $4,500 million in 2021 and is projected to grow to $15,800 million by 2030. The data produced by vehicle manufacturing, as well as in-vehicle systems, produces a vast amount of informational data per vehicle, which presents highly lucrative opportunities if this data can be harnessed and analyzed properly. Data marketplaces in the automotive industry help vehicle manufacturers, dealerships, and other industry stakeholders take advantage of this data through:
- Monetizing vehicle data. Because vehicles produce so much useful data, such as from internet-connected vehicles to digital service ecosystems, it’s a no-brainer for automotive businesses to harness this data and monetize it. The main role of a car data marketplace is to facilitate data sharing between the car manufacturer and a third-party service provider via API integrations. These unified APIs make reduce the complexity for third parties that need to make their apps and services work properly in vehicles from various manufacturers. Further, an automotive data marketplace can also handle the business side of things, providing a business and legal framework where third parties can avoid contracting directly with car manufacturers.
- Development of better vehicle features. Taking advantage of connected car data, vehicle manufacturers can potentially develop new vehicle features or improve existing ones to provide better value to consumers. This helps the vehicle manufacturer zero in on aspects that can differentiate their offering to their target market, potentially giving them an edge over their competitors.
- Improving different aspects across the automotive supply chain. OEMs, suppliers, insurers, and even government and regulatory bodies benefit from data sharing via an automotive data marketplace. For instance, OEMs use car data analytics to better understand how customers are using their vehicles and improve the link between dealerships and customers. Insurers use data to optimize their pricing for occasion-based policies and offer usage-based insurance contracts based on an extended understanding of consumer behavior. Governments and regulatory bodies set the standards for the collection and sharing of car data, ensuring security and privacy are maintained. Further, they can mandate certain services in the name of public good, such as emergency call features.
Transportation
In the world of transportation, optimizing logistical processes to ensure that the distribution of products is as efficient as possible is paramount. Because of the accessibility behind data marketplaces, transportation organizations can take advantage of data to create systems that get goods to consumers faster and for less cost. Data marketplaces can help transportation organizations with these initiatives and more by:
- Providing route optimizations. Effective routing is a big challenge in logistics, but worth the effort to optimize. While internal routing data can be used to determine better routes over time, augmenting that information with external real-time data, including weather information, traffic, and delivery sequences, can assist with creating the most efficient logistical plans. For example, last-mile delivery (meaning the last mile that is needed to get a product to a customer) has long been a challenge for delivery companies. Finding cost-effective ways to get goods from a central location across the last leg of the journey can be difficult, but crowdsourcing applications using big data (which can be facilitated using a data marketplace) connect delivery drivers with the opportunity to deliver more packages on their everyday route, making deliveries more efficient.
- Enhancing road safety. Data from millions of connected vehicles on the road can be utilized for initiatives like enhancing road safety. The Eastern Transportation Coalition (TETC) is an example of a data marketplace that provides the opportunity for Department of Transportation agencies and other public agencies in the United States to access connected vehicle data, such as vehicle speeds, journey information, hard braking events, and more to use for initiatives like reducing traffic congestions, improving air quality (through more efficient public transport, for instance) and otherwise preserving transportation assets.
- Providing real-time transportation market intelligence data. Transportation organizations want to be more data-driven in their decision-making. To do this effectively, they want access to real-time granular data surrounding transportation management. This transportation market intelligence data, which consists of a wide number of assets, like transport pricing (plane, truck, boat, and temperature-controlled, palletized, and bulk), contract data, and spot market data needs to be accessible, which a data marketplace can facilitate. This information helps transportation organizations recognize and plan for the future and otherwise make proactive decisions. In addition, it tells them where they stand against competitors in terms of efficiency and spending.
Data Storage Options
There are a variety of approaches to storing datasets. In this section, we’ll describe two of those storage options, data lakes and data warehouses.
| Data Lake | Data Warehouse |
| Stores structured, unstructured, semi-structured, or raw data that doesn’t yet have a defined purpose | Stores structured, filtered data that has already been processed for a specific purpose |
| Data scientists are usually the types of professionals that access data lakes | Business professionals typically access data stored in a data warehouse |
| Because a data lake doesn’t have a defined structure, it is easy to access and change | Data warehouse architecture is more structured, which makes data easier to understand, but also makes changes to the architecture more difficult |
Data lakes in general have become a more popular option over data warehousing, due to the fact that businesses are processing higher volumes of data than ever before. Because of its flat architecture, data lakes can store large volumes of data without the need to process it all immediately, making it easy for data scientists to pull from when needed, and making for easier accomodation for storage of big data.
Databricks Delta Lake is one example of a data lake storage solution. It is an open-source advanced storage layer in the Databricks lakehouse platform—a platform that combines the best of data warehouses and data lakes. With Databricks Delta Lake, there are several benefits, including:
- Incremental scale processing using only a single data copy for streaming and batch operations
- Data shares can be faciliated without the need to create a compute pattern beforehand
- Datasets as large as a terabyte can be shared through cloud sotrage systems, such as ADLS, Amazon S3, and GCS
Common Data Marketplace Platforms
There are several existing data marketplace platforms that have made a name for themselves in the data fulfillment industry, whether it’s due to reputation, availability of data, or otherwise. Some marketplaces specialize in providing specific data for an industry, such as finance. The data marketplaces listed in the table below represent some of the most well-known offerings.
| Data marketplace | Functionality and features |
| Snowflake data share |
|
| AWS Data Marketplace (a Marketplace/Exchange hybrid) |
|
| Google Data Marketplace |
|
| Informatica Cloud Data Marketplace |
|
Data Marketplace vs. Data Exchange
In the data fufilment industry, a data marketplace is typically used to describe an online platform (often cloud-based) that facilitates buying and selling of data. Data on these platforms are packaged into products, so you can purchase them much like you would a product from Amazon or another e-commerce website.
On the other hand, a data exchange is an option for organizations where, instead of selling their data, find value in exchanging their data with another organization (often in the same industry, but not always). Instead of paying for data, organizations gain mutual benefits by exchanging data with each other.
More nuanced differences between data marketplaces and exchanges are explored in the following table:
| Data marketplace | Data exchange |
| Money is exchanged for a data product | Data is made available on the data exchange platform, so a one-to-one exchange isn’t typically necessary, but participants are expected to provide datasets regularly |
| Can be leveraged by organizations of all sizes | Can be leveraged by organizations of all sizes, but the most valuable players are typically more data-mature organizations |
| Data can be made public (available to anyone) or private (available to specific members of a group), but it’s typically more beneficial to make data as publicly available as possible | Data can be made public (examples of public data exchanges are AWS and Google Cloud Analytics Hub) or private (useful for situations where a select group of companies want to share data with each other only) |
| More one-sided, where a data provider is providing a data product for sale, and the buyer simply purchases it | Available datasets provide mutual value to members of the data exchange |
It’s important to note that over time, the terms data marketplace and data exchange have become somewhat synonymous because the technologies surrounding them are similar. In some cases, such as with the Revelate data marketplace, both buying and selling of data as well as an exchange of data can be performed. Different front-facing environments can be prioritized based on user, so for instance, a public user may see a data product for sale, while a private user that’s part of an established data exchange network may see the same data product readily available for them to download at no charge. In this way, data fulfillment (whether it’s being bought, sold, or shared) can be controlled on one centralized platform.
Your Revelate data marketplace experience is highly customizable in terms of data cataloging, segmenting, and marketing data products to suit your target audience(s). Revelate facilitates a seamless experience in terms of purchasing, selling, sharing, or exchanging data products, strengthening the relationship between data providers and customers. Get Started
Challenges of Data Marketplaces
1. Data Marketplaces Standardizations Limit Display Flexibility
In an effort to keep data products consistent and displayed homogeneously, marketplace owners enforce a strict set of standardizations that all sellers who list on marketplaces must adhere to. For example, in niche data marketplaces, metadata of a data product will only be listed as it pertains to the niche itself (based on industry, region, subject matter, etc.) while in more broad marketplaces, different metadata is listed for the same product. Since the operator owns the marketplace, they have the authority and the right to determine how a product is listed. Products listed on marketplaces might not be conveyed in a way in which the buyer can make sense of the data, understand its value, and be enticed enough to make a purchase.
This poses problems for businesses that want to monetize unstructured data sets, create custom data products, or experiment with different pricing models. Even though a marketplace provides a centralized platform that in theory should simplify the comparing and contrasting of data products, this is not always the case. Data products cannot be watered down to a certain set of predefined criteria because each product can be completely different from the next.
For ‘irregular’ types of products to be searchable, each product needs its own metadata and sometimes even a separate category. Comparably, listing a vast array of metadata that is not fit-for-purpose isn’t a viable solution either. Listing too much irrelevant metadata makes the buyer’s journey one that is tedious and burdensome. Unfortunately, marketplaces are, for the most part, not flexible enough to accommodate variances in products, nor are they able to display a detailed data catalog of the provider’s full range of available offerings in the way that is best suited for the data, which contributes to the difficulties for buyers to find and procure the specific data they need.
2. Tailoring Data Products for Display on Different Marketplaces
Building off the fact that data marketplaces have different data display standardizations, it’s the responsibility of the data provider to tailor their data products to fit these standardizations. To maximize reach and increase the ROI of a data product, it’s often necessary to list the product on multiple data marketplaces, meaning that the data provider needs to consider that product display on one marketplace won’t be as effective as on another.
3. Choosing Which Data Marketplaces to Use
With hundreds of data marketplaces to choose from, it can be difficult to choose the right data marketplaces to list your datasets with. While one data marketplace might have the display functionality that presents your dataset well, another may have a more well-established client base that could potentially have your dataset reach more buyers.
Simply listing on every relevant data marketplace might not be the best option either, since data transfer costs could end up costing your organization more than the sale of the data product itself. Choosing the right set of marketplaces to list datasets, then, is often a process of trial and error, which starts with an extensive research process to determine which data marketplaces are worth the initial time investment.
4. The Consumer Journey is Often Convoluted
There is a seemingly infinite amount of data available to find on the internet, but this doesn’t mean that all data can be easily discovered or accessed. It also doesn’t mean that the specific data a buyer is looking for is available. Data discovery is a step that is often overlooked by providers that list on marketplaces. Providers may falsely assume that discovering their products is easy on marketplaces because the marketplace already has existing buyers within their target market.
Although marketplaces do connect buyers and sellers, data discovery is usually not a straightforward process when the seller has limitations on what they can and cannot do in a marketplace. Marketplaces can either be private or public as well. In private marketplaces, both the buyer and seller need approval by the platform owner before they can join, which adds another layer of friction for the seller and buyer to access the data they seek. For marketplaces in the public domain, buyers can freely exchange data with any seller, provided they can determine whether the data source is reliable and trustworthy.
Implied trust between the seller and the buyer. Simply put, trust is an implied element of marketplaces for both sellers and buyers (even though there’s a degree of due diligence they likely do go through before actually making a purchase). Buyers need to trust that the data is of high quality and that the provider is a reliable source. On the other hand, sellers need to trust that the buyer, who might not necessarily be the consumer, will adhere to the licensing agreement and usage rights in addition to the security and privacy restrictions of the data after purchasing.
Irrespective of whether a marketplace is public or private, the seller is not the operator. This means that the data producer does not have ultimate control over how their products are displayed, found or accessed by the potential buyer, which can lead the buyer into the hands of a competitor—whether that competitor is another marketplace with better delivery mechanisms and terms of use, or another data provider themselves.
5. Listings Have a High Degree of Competition
B2B data marketplaces remain competitive against other marketplaces by keeping their prices low and usually at a fixed cost. Although a marketplace creates a favorable environment for buyers to access data at a relatively low price point, the sellers remain disadvantaged by being unable to experiment with different offerings and price points based on how their data is packaged.
Marketplaces also provide a means for data providers to compete directly with one another. Since products are standardized by the marketplace owner in terms of metadata and search filters, identical data products from separate providers cannot list their products in the way they should to differentiate themselves from the competition. Similarly, the order in which a listing appears after a set of filters is applied can affect whether or not a buyer decides to purchase a data set.
The marketplace owner is the only authoritative body that can determine whether your listing will take the top spot or whether your data set will be buried on another page, leaving the door wide open for the competition to likely accumulate the bulk of the profits.
6. General Lack of Trust in Data Marketplaces
Trust is critical to the success of a data marketplace. If the marketplace can’t establish trust with it’s data providers and consumers, then the whole operation quickly falls apart. The main challenges with establishing trust surround data product quality and security, and at a lesser extent, non-standard pricing and long data license commitments, the latter putting more risk on the consumer in terms of receiving data they can’t use.
For a data marketplace to be successful, they have to overcome the hurdle of trust by compiling a variety of trusted data providers, and, like other eCommerce businesses, have reliable user authentication and verification, and a clear authority that data providers and consumers can turn to to help resolve conflicts.
The Evolution of the Data Marketplace
Every data marketplace strives to connect data providers to potential buyers. Due to the limitations of traditional marketplaces, including those outlined in the previous section, there has been a need for data marketplaces to evolve. The vast majority of the issues with data marketplaces stem from the control being placed in the hands of the operator, not the data seller. One way to counteract this is to offer a data marketplace with more flexibility for the data seller, but in some cases, it may be beneficial for the data provider to be the sole owner and operator of the data marketplace.
With the challenges of the traditional data marketplace in terms of product display limitations, search constraints, and more, it makes sense that the data marketplace would be iterated on over time, leading to a more effective solution that works for more organizations that want to monetize or share their data.
That’s where Revelate comes in. As a holistic data fulfillment platform, Revelate can minimize or remove the nuances associated with the traditional data marketplace, including:
1
Creating standardizations based on the consumer’s purpose. Every data offering can be customized—however broad or niche. Derivative products can be created that suit the buyer’s purpose, and the data provider can freely experiment with different pricing options. With a robust data fulfillment platform, the data provider has the power to produce a data catalog that includes custom data products and/or unstructured data. Standardization using metadata for enhanced discovery and accessibility is also prioritized through Revelate’s fully managed, white-label data web store, ensuring the data product is fit for purpose.
2
Streamlining the consumer’s journey. Revelate’s data web store allows full control of the consumer’s journey from data discovery to deliverability based on the granularity of the data products. As a data provider, you can offer complete data traceability to foster a stronger consumer relationship based on trust. This will create a continuous feedback loop that you can use to further enhance discoverability and/or build better data products that will sell in larger volumes. There is also no direct competition on the same platform. Each data web store is its own entity—your data products will not be in direct competition with similar data products offered by other providers. You choose how the data is displayed and optimize the listing as needed.
3
Leveraging data marketplaces and data web stores. Minimizing friction in the buyer’s journey from discovery to procurement to access positively impacts any organization that monetizes its data. Leveraging data marketplaces and data web stores can make this happen by getting the best of both worlds: using an existing customer base to immediately reach potential buyers and amplifying this reach by controlling the consumer’s experience and listing products in a way that entices the consumer to make a purchase.
One example that illustrates how effective a data web store can be is Revelate’s own customer, CME Group. As the world’s leading derivatives marketplace, CME Group provides over 600 terabytes of historical, alternative, and analytic datasets via its extensive modern data marketplace called CME DataMine.
But it wasn’t always this easy for them to efficiently and effectively sell their data.
When CME Group approached Revelate, they didn’t have a standard system for handling and processing data. Data was dispersed across multiple departments, and normalizing, centralizing, and extracting the data to make it easily accessible online, while maintaining security and privacy, was a huge challenge. In the interim, CME sold a single file with the entirety of their feed, which meant that only the customers with the resources to extract needed information (i.e., large, enterprise organizations) could effectively use their data, even though small and mid-size businesses could absolutely benefit from CME’s data as well. The method of selling a singular dataset, in essence, was not only providing not the most effective experience for their existing customers but it was also unintentionally alienating an entire potential group of customers.
Traditional data marketplaces, of course, could be utilized, but within their limitations. Given the fact that CME provides a ton of data in all different configurations for many different industries and use cases, disseminating data products down to a standardized view wouldn’t provide an effective customer experience—finding and purchasing the right datasets simply wouldn’t be easy or effective.
The solution? A fully-customizable, flexible, modern data web store.
With a data web store, CME Group was able to:
- Make data easier to sell. Getting more customers to purchase their data products was a top priority for CME Group, but attracting more customers without first having a more effective way to sell them data wouldn’t be a good strategy for customer retention. Making data easier to sell with an easy-to-use platform would, in turn, have the effect of generating more customers.
- Unlock the value of granular data. With previously selling a singular data file, streamlining the process of providing granular data files, and determining the value of those files to different target data customers, needed to happen. Using their new data marketplace, CME Group is able to market their granular data sets effectively by organizing them into groups and using sales data to determine the correlation between data set granularity and usefulness for different customers and stand out from other market providers.
- Price data products effectively. Pricing data products can be a complex task, especially when offering more granular datasets. Advice from Revelate helped CME Group set effective initial pricing.
- Stand out as a data provider. Having full control over the customer buying journey and being able to provide granular data products helped CME Group reach a wider data consumer audience.
- Future-proof data monetization. CME Group’s data web store isn’t a static entity; as technological advancement opportunities present themselves, the store can be upgraded, changed, and enhanced to keep up with future developments.
A data web store created with Revelate is a centralized data commerce platform that makes cataloging, segmenting, and marketing data products externally effortless for enterprises across any industry or location. It securely consolidates, ingests, and aggregates large amounts of data and allows providers to turn data into products that buyers can readily access with the least amount of friction. In contrast to a traditional marketplace for data, Revelate’s data web store empowers providers to curate the data consumer’s end-to-end experience, strengthening the relationship between the two parties.
Leveraging Revelate’s data web store in conjunction with another data marketplace if the data provider so wishes can mobilize data—i.e., provide a way for data producers to efficiently package and distribute data while enabling consumers to access said data easily and quickly.
Buying vs. Building a Data Marketplace Platform
Building a modern data marketplace (e.g., data web store) is one solution to the dilemmas of traditional data marketplaces, but there are also times when an existing data marketplace may provide enough lucrative opportunities for a data provider that they consider purchasing it, alongside its existing client base, and running it themselves.
Managing and maintaining a data marketplace requires major investment, a high level of data maturity and sophistication, product development, and runtime experience, plus infrastructure and scalability expertise, and the ability to manage and deliver potentially high-risk data products. In short, a data marketplace is expensive to run and build but has a high potential for profitability.
Let’s compare the differences between buying vs building a data marketplace:
| Building a Data Marketplace | Buying or Licensing a Data Marketplace |
| Allows complete customization of the experience from the ground up | Will have to work with the existing infrastructure, but speed to market is much faster |
| The marketplace provider can provide features and benefits to consumers that set them apart from competing marketplaces | Provides access to an existing user base, which may be easier to maintain than building a new one |
| With complete ownership and control custom whitelabeling and licensing options are possible | Existing data marketplaces are likely to have already dealt with and solved licensing, compliance, and governance issues |
| Once the costs of building the marketplace are recouped, potential for higher revenue generation is possible because profits don’t have to be shared with other vendors, plus the marketplace owner has more control over licensing, pricing, and distribution of data products | While margins are perhaps less, access to an established marketplace with expertise, clients, reputation, and integrations for data transfer is easier to hit the ground running with compared to building a solution from scratch |
Additional considerations for buy vs build when it comes to data marketplaces include:
- Platform scalability
- Data security
- Support and maintenance
Conclusion
Since the start of the internet, data marketplaces companies have emerged to provide a way to connect data providers and consumers. Like any technology, it makes sense that data marketplaces have evolved over the years in an effort to keep up with the changing needs of data providers and consumers, including the development of a variety of different types of modern data marketplaces (IoT, open data marketplaces, etc.) to continue allowing data to be provided and consumed.
At the same time, it’s no surprise that challenges have emerged with data marketplaces. While these challenges can be difficult to face, it’s imperative that they are met with creative solutions, especially as big data becomes more prevalent, and the ability to consume and use big data becomes easier for organizations.
Different types of data fulfillment that an organization would want—a data marketplace, data exchange, and data sharing network don’t have to occur on different platforms. Revelate’s fully managed and white-label data fulfillment platform can handle the access permissions and security you need to utilize one platform for all your data fulfillment needs. What’s more, the platform has robust data automation solutions so that manual management of data orders doesn’t burden your IT department.
Unlock Your Data’s Potential with Revelate
Revelate provides a suite of capabilities for data sharing and data commercialization for our customers to fully realize the value of their data. Harness the power of your data today!
