New Directions
Collectively, the data team provides the essential capabilities an enterprise needs: SQL understanding to manage cloud data warehouses, the ability to deploy and secure cloud infrastructure, familiarity with data orchestration and pipelines, and a strong understanding of applying business logic through transformations. These tasks remain the foundation of data management.
Once those needs are met, enterprises can look ahead to new, advanced use cases. Here are several of the exciting avenues data teams can explore to improve established processes and start new initiatives.
-
- Enable data democratization. Business professionals need to become data literate and data-driven, and they’ll be expected to acquire and demonstrate basic competency with technical tools. With that knowledge, they can accomplish tasks they would previously have asked IT to perform for them, such as preparing a data set for analytics.
-
- Improve data governance. Building a modern data team is a great opportunity to structure better data governance. This team can work together to establish data governance processes to maintain data security, such as creating an audit trail for data access and ensuring that only approved data is loaded into the data warehouse, data lake, and downstream tools.
-
- Perform advanced analytics. With centralized data and faster data transformation workflows, enterprises can launch advanced projects such as augmented analytics for data preparation, data discovery, and data science to unlock actionable insights.
- Deliver a better customer experience. Empowering business users with customer experience data such as sentiment analysis or telemetry information can help organizations determine new ways to delight customers, detect dissatisfaction, and create new products based on customer insights.
These trends are driving change in today’s data terms. The highly technical coding skills and hands-on task that were in high demand ever a few years ago just to keep workflows going are giving way to low-code and no-code tools. Data engineers will always be a critical part of any modern data team, but the kind of hand-coding that was once routine is nearly impossible with today’s data volumes.
Who’s Who on a Modern Data Team
To tackle the challenges ahead, companies need to rethink the structure of their data teams.
What roles and responsibilities make up the modern data team? The job titles will vary depending on the business and industry, but each team member’s responsibilities fall into one of the following five categories.
1. A technical role — a data engineer or an ETL developer — builds workflows. This person will be responsible for making sure data pipelines and ETL jobs are running and colleagues have the data access they need for their projects. This team member oversees data migration, orchestration, and transformation and helps develop applications and systems that drive advanced analytics use cases, including AI and machine learning.
2. Data scientists are necessary to derive value and insight from data. Although data scientists are normally situated inside IT, nowadays it’s normal to find them in other parts of the business. Data scientists find innovative ways to work with data and help teams achieve a rapid ROI on analytics efforts using methods including data curation or advanced search, matching, and recommendation algorithms. Data scientists need access to the highest quality of data and large amounts of computing resources to extract deeper insights. Using the data scientists’ time for sourcing, preparing, and checking data in the warehouse is wasteful.
3. A data analyst queries and reports on data in the data lake or cloud data warehouse. Analysts can use these findings to build interactive charts and dashboards for business user reporting, diagnostics, and decision making.
4. Citizen data professionals (CDPs) are increasingly important members of the team as the demand for data scientists is often unmet due to a skills shortage. A CDP is a data-savvy knowledge worker who sits outside the IT department and wants to tackle data problems using intuitive, simple tools to load and centralize data sources for analytics and innovation. CDPs do not replace but rather complement technical team members in solving business challenges because they know their data sources better than anyone and can work with a technical counterpart to troubleshoot and enhance the use of data and insights.
5. An executive team member — be it a CTO, CIO, or CDO — keeps the data team on track and shapes the companywide data management strategy. This person will work across disciplines to ensure that every department’s initiatives are aligned and business logic, governance, and security are properly managed.