Data management

The data lifecycle

Data management comprises all disciplines related to handling data as a valuable resource, it is the practice of managing an organization's data so it can be analyzed for decision making.

Concept

The concept of data management emerged alongside the evolution of computing technology. In the 1950s, as computers became more prevalent, organizations began to grapple with the challenge of organizing and storing data efficiently. Early methods relied on punch cards and manual sorting, which were labor-intensive and prone to errors. The introduction of database management systems in the 1970s marked a significant milestone, enabling structured storage and retrieval of data.

By the 1980s, relational database models revolutionized data management, emphasizing the importance of data as an asset and fostering a data-centric mindset in business. This era also saw the rise of data governance practices, which prioritized the organization and regulation of data to ensure quality and compliance. Over time, advancements in technology, such as cloud computing and big data analytics, have further refined data management, making it a cornerstone of modern business operations.

Today, data management encompasses a wide range of practices, from data storage and security to analytics and decision-making, reflecting its critical role in driving innovation and efficiency across industries.

Topics in Data Management

The Data Management Body of Knowledge (DMBoK), developed by the Data Management Assocation International, outlines key knowledge areas that serve as the foundation for modern data management practices. These categories provide a comprehensive framework for organizations to manage data as a strategic asset.

Data Governance

Main articles: Data governance, Data steward, Data strategy, Data ethics, and Data custodian

Setting policies, procedures, and accountability frameworks to ensure that data is accurate, secure, and used responsibly throughout the organization.

Data Architecture

Main articles: Data Architecture and Data mesh

Focuses on designing the overall structure of data systems. It ensures that data flows are efficient and that systems are scalable, adaptable, and aligned with business needs.

Data Modeling and Design

Main articles: Data modeling and Database design

This area centers on creating models that logically represent data relationships. It’s essential for both designing databases and ensuring that data is structured in a way that facilitates analysis and reporting.

Data Storage and Operations

Main articles: Database administration and Database management system

Deals with the physical storage of data and its day-to-day management. This includes everything from traditional data centers to cloud-based storage solutions and ensuring efficient data processing.

Data Security

Main articles: Data Security and Data erasure

Encompasses the practices and technologies needed to protect data from unauthorized access, breaches, and other security threats, ensuring data privacy and compliance with regulations.

Data Integration and Interoperability

Main articles: Data integration; Extract, transform, load; Extract, load, transform; and Dataflows

Ensures that data from various sources can be seamlessly shared and combined across multiple systems, which is critical for comprehensive analytics and decision-making.

Document and Content Management

Main articles: Document management system, Content management, and Records management

Focuses on managing unstructured data such as documents, multimedia, and other content, ensuring that it is stored, categorized, and easily retrievable.

Data Warehousing, Business Intelligence and Data Analytics

Main articles: Data warehouse, Business intelligence, data mart, Data analytics, Data analysis, data mining, and Data science

Involves consolidating data into repositories that support analytics, reporting, and business insights.

Metadata Management

Main articles: Metadata management, Metadata, Metadata registry, Metadata discovery, and Metadata publishing

Manages data about data, including definitions, origin, and usage, to enhance the understanding and usability of the organization’s data assets.

Data Quality Management

Main articles: Data quality, Data quality assurance, Data integrity, Data cleansing, and Data discovery

Dedicated to ensuring that data remains accurate, complete, and reliable, this area emphasizes continuous monitoring and improvement practices.

Reference and master data management

Main articles: Reference data and Master data management

Reference data comprises standardized codes and values for consistent interpretation across systems. Master data management (MDM) governs and centralizes an organization’s critical data, ensuring a unified, reliable information source that supports effective decision-making and operational efficiency.

Data security

Main articles: Data security and Data access

Data security refers to a comprehensive set of practices and technologies designed to protect digital information and systems from unauthorized access, use, disclosure, modification, or destruction. It encompasses encryption, access controls, monitoring, and risk assessments to maintain data integrity, confidentiality, and availability.

Data privacy

Main articles: Data privacy and Data subject

Data privacy involves safeguarding individuals’ personal information by ensuring its collection, storage, and use comply with consent, legal standards, and confidentiality principles. It emphasizes protecting sensitive data from misuse or unauthorized access while respecting users' rights.

Usage

The distinction between data and derived value is illustrated by the information ladder. However, data has staged a comeback with the popularisation of the term big data, which refers to the collection and analyses of massive sets of data. While big data is a recent phenomenon, the requirement for data to aid decision-making traces back to the early 1970s with the emergence of decision support systems (DSS). These systems can be considered as the initial iteration of data management for decision support.

Several organisations have established data management centers (DMC) for their operations.

Data sources

Marketers and marketing organizations have been using data collection and analysis to refine their operations for the last few decades. Marketing departments in organizations and marketing companies conduct data collection and analysis by collecting data from different data sources and analyzing them to come up with insightful data they can use for strategic decision-making (Baier et al., 2012). In the modern business environment, data has evolved into a crucial asset for businesses since businesses use data as a strategic asset that is used regularly to create a competitive advantage and improve customer experiences. Among the most significant forms of data is customer information which is a critical asset used to assess customer behavior and trends and use it for developing new strategies for improving customer experience (Ahmed, 2004). However, data has to be of high quality to be used as a business asset for creating a competitive advantage. Therefore, data governance is a critical element of data collection and analysis since it determines the quality of data while integrity constraints guarantee the reliability of information collected from data sources. Various technologies including Big Data are used by businesses and organizations to allow users to search for specific information from raw data by grouping it based on the preferred criteria marketing departments in organizations could apply for developing targeted marketing strategies (Ahmed, 2004). As technology evolves, new forms of data are being introduced for analysis and classification purposes in marketing organizations and businesses. The introduction of new gadgets such as Smartphones and new-generation PCs has also introduced new data sources from which organizations can collect, analyze and classify data when developing marketing strategies. Retail businesses are the business category that uses customer data from smart devices and websites to understand how their current and targeted customers perceive their services before using the information to make improvements and increase customer satisfaction (Cerchiello and Guidici, 2012). Analyzing customer data is crucial for businesses since it allows marketing teams to understand customer behavior and trends which makes a considerable difference during the development of new marketing campaigns and strategies. Retailers who use customer data from various sources gain an advantage in the market since they can develop data-informed strategies for attracting and retaining customers in the overly competitive business environment. Based on the information on the benefits of data collection and analysis, the following hypotheses are proposed: The sources of data used as the foundation of data collection and analysis have a considerable impact on the data analysis tools used for analyzing and categorizing data.

Data analysis tools

Organizations use various data analysis tools for discovering unknown information and insights from huge databases; this allows organizations to discover new patterns that were not known to them or extract buried information before using it to come up with new patterns and relationships (Ahmed, 2004). There are 2 main categories of data analysis tools, data mining tools and data profiling tools. Also, most commercial data analysis tools are used by organizations for extracting, transforming and loading ETL for data warehouses in a manner that ensures no element is left out during the process (Turban et al., 2008). Thus the data analysis tools are used for supporting the 3 Vs in Big Data: volume, variety and velocity. Factor velocity emerged in the 1980s as one of the most important procedures in data analysis tools which was widely used by organizations for market research. The tools used to select core variables from the data that was collected from various sources and analyzed it; if the amount of data used to be too huge for humans to understand via manual observation, factor analysis would be introduced to distinguish between qualitative and quantitative data (Stewart, 1981). Organizations collect data from numerous sources including websites, emails and customer devices before conducting data analysis. Collecting data from numerous sources and analyzing it using different data analysis tools has its advantages, including overcoming the risk of method bias; using data from different sources and analyzing it using multiple analysis methods guarantees businesses and organizations robust and reliable findings they can use in decision making. On the other hand, researchers use modern technologies to analyze and group data collected from respondents in the form of images, audio and video files by applying algorithms and other analysis software Berry et al., 1997). Researchers and marketers can then use the information obtained from the new generation analysis tools and methods for forecasting, decision support and making estimations for decision making. For instance, information from different data sources on demand forecasts can help a retail business determine the amount of stock required in an upcoming season depending on data from previous seasons. The analysis can allow organizations to make data-informed decisions to gain competitive advantage in an era where all businesses and organizations are capitalizing on emerging technologies and business intelligence tools to gain competitive edges. While there are numerous analysis tools in the market, Big Data analytics is the most common and advanced technology that has led to the following hypothesis: Data analytic tools used to analyze data collected from numerous data sources determine the quality and reliability of data analysis.

Data security and data privacy

While organizations need to use quality data collection and analysis tools to guarantee the quality and reliability of the customer data they collect, they must implement security and privacy strategies to protect the data and customer information from privacy leaks (Van Till, 2013). A study conducted by PWC indicated that more than two-thirds of retail customers prefer purchasing products and services from businesses that have data protection and privacy plans for protecting customer information. Also, the study indicated that customers trust businesses that can prove they cannot use customer data for any other purposes other than marketing. As technology and the Internet continue improving, the success of businesses using it as a platform for marketing their products will depend on how effectively they can gain and maintain the trust of customers and users. Therefore, businesses will have to introduce and implement effective data protection and privacy strategies to protect business data and customer privacy. Although developing trust between customers and businesses affects the customers’ purchasing intentions, it also has a considerable impact on long-term purchasing behaviors including how frequently customers purchase which could impact the profitability of a business in the long run. Thus, the above information leads to the following hypothesis: Implementing data security and privacy plans has a positive impact on economic and financial outcomes.

Financial and economic outcomes

Studies indicate that customer transactions account for a 40% increase in the data collected annually, which means that financial data has a considerable impact on business decisions. Therefore, modern organizations are using big data analytics to identify 5 to 10 new data sources that can help them collect and analyze data for improved decision-making. Jonsen (2013) explains that organizations using average analytics technologies are 20% more likely to gain higher returns compared to their competitors who have not introduced any analytics capabilities in their operations. Also, IRI reported that the retail industry could experience an increase of more than $10 billion each year resulting from the implementation of modern analytics technologies. Therefore, the following hypothesis can be proposed: Economic and financial outcomes can impact how organizations use data analytics tools.

See also

• Open data
• FAIR data
• Pseudonymization
• Information architecture
• Enterprise architecture
• Information design
• Information system
• Controlled vocabulary
• Data curation
• Data retention
• Data Management Association
• Data management plan
• Data mesh, a domain-oriented data architecture
• Computer data storage
• Data proliferation
• Digital preservation
• Document management
• Enterprise content management
• Hierarchical storage management
• Information repository
• Machine-readable document
• Performance report
• System integration
• Customer data integration
• Identity management
• Identity theft
• Data theft
• ERP software
• CRM software