How to support accurate revenue forecasting with data science and dataops

Data scientists and technologists responsible for data governance, engineering, and integration should look for opportunities to use data analytics and AI for strategic decision-making. Finance, marketing, and sales departments all have important use cases, such as tracking cash flow, managing advertising campaigns, and prioritizing sales prospects.

One area that interests all business leaders is revenue forecasting, as all departments provide inputs into revenue forecasting models and manage budgets that depend on them. However, accurately forecasting revenue is a significant challenge. In the 2024 Sales Forecasting Benchmarking Report, 43% of respondents said their sales forecasts were typically off by 10% or more; 38% reported data quality issues; and 35% said the forecasting process took too long.

“Forecasting is essential for the financial success of every organization, but it’s often a significant challenge,” says Arnab Mishra, CEO of Xactly. “Sales and finance teams encounter common obstacles when making forecasts, including reporting systems that lack access to historical CRM or performance data and uncertainty about where the pipeline data is from. The most successful organizations have revenue and finance leaders who integrate innovative forecasting technology solutions and prioritize accurate forecasts.”

Enterprises typically staff financial planning and analysis (FP&A) professionals responsible for developing revenue forecasting models, dashboards, reports, and recommended actions. Public companies must follow the SEC financial reporting guidelines and regulations, often utilizing specialized financial reporting tools, leveraging machine learning models, and creating multiple forecasts. Smaller organizations may prefer using rules-driven approaches and self-service business intelligence tools to develop their projections.

Data professionals must develop their business acumen and partner with FP&A professionals as customers. Understanding their goals and partnering with them in their data, modeling, analytics, and forecasting objectives can help data teams deliver significant business value in their organizations.

Steps for forecasting revenue

Forecasting revenue is done at a macro level when companies seek guidance, and public companies publish their projections for the upcoming quarters and years. Many companies will also forecast revenue for each business unit, product line, and geography.

Forecasting revenue involves the following steps, which FP&A professionals usually perform during the budgeting process, at strategic decision-making points, and when there are major business changes or external factors.

1. Data is gathered from internal sources, including ERPs, CRMs, marketing automation platforms, and customer service tools, as well as external sources, including information on economic factors, customer demand, regulatory changes, climate forecasts, and political factors.
2. Time periods are selected for the analysis to factor in what data sets and segments are needed.
3. External factors, constraints, and other risks that might accelerate or inhibit growth are reviewed, including the business’s supply chain factors, strategic decisions, labor conditions, and other global and local events.
4. Tools and forecasting models are selected to forecast revenue from existing customers, revenue sources, and new customer orders. Modeling revenue may use a top-down approach to forecast changes to existing revenue and a bottom-up approach for factoring in the sales pipeline. Multiple models are often created to consider different planning scenarios and benchmarked against external forecasts.
5. Finalized models are presented to executives, plugged into the ERP, CRM, and other planning tools for aligning resources, and monitored for accuracy.

While the steps are fundamentally simple, making accurate forecasts has many challenges. There are several ways data and analytics teams can support this process.

Having clean and centralized data is a prerequisite

Centralizing, cleansing, and integrating data, and addressing data debt are primary responsibilities of dataops and data governance professionals. Without these disciplines, FP&A professionals lose trust in the company’s data sources and spend more time wrangling data, which takes away from their focus on developing accurate models.

“With the rise of AI, most organizations are working to understand how to get more value from their systems and data,” says Grant Peterson, chief product officer at Conga. “The keys to getting the maximum value include the extent and organization of data, the ability to integrate additional data sources, and the ability of tools and systems to provide these. Organizations lack a holistic solution to collect data to support decisions and drive revenue accurately.”

Data professionals should consider the following strategies for providing the best quality data that’s easy for FP&A professionals to utilize:

• Large enterprises may want to invest in data fabrics to help centralize access to multiple enterprise and SaaS platforms.
• Dataops teams should develop robust, real-time data pipelines and select a subset of data quality metrics to optimize.
• Data governance leaders should ensure data catalogs are up to date and meet the needs of FP&A professionals.
• Data and analytics leaders should establish governance on citizen data science programs and consider FP&A professionals as key participants.
• Data scientists should partner with FP&A professionals in their modeling efforts by establishing guidelines and standards in agile data practices, design thinking in data science, and modelops.

The challenges of forecasting growth

Data professionals should also learn some of the challenges that FP&A professionals face, especially when forecasting growth. These forecasts require joining and modeling data around sales pipelines, supply chains, and economic factors in transparent ways and with believable results.

Dataops and data governance leaders should consider FP&A key stakeholders in identifying data quality issues, as forecasting often requires additional data quality considerations and data lineage practices. For example, using spreadsheets to fix data issues is error-prone, delays forecasting, limits collaboration, and creates transparency issues. Forecasts relying on sales data require reviewing the timeliness, accuracy, and other data quality issues stemming from how and when sales professionals work in their CRM.

“Data quality plays a big role in revenue forecasting, especially when it comes to predicting growth,” says Steve Smith, global director of strategic projects at Esker. “While forecasting existing revenue is straightforward, relying on past sales forecasts for future growth can be problematic due to potential biases or incomplete data. Additionally, complex sales cycles that require multiple sign-offs and market volatility can further disrupt timing and accuracy in order predictions.”

Forecasting must also consider factors that are external to the organization and leverage third-party data sources for economic, customer, and other trends. To enable growth forecasting, it is important to evaluate, profile, and integrate new data sources, including unstructured ones such as news sources.

“Forecasting models traditionally rely heavily on internal data sources, such as marketing expenditures and customer counts,” says Krishnan Venkata, chief client officer at LatentView. “Though these internal metrics are crucial, they often fail to incorporate external data inputs that could significantly impact forecasts.”

Venkata of LatentView recommends incorporating external trend data for consumer interest shifts, social media for real-time sentiment analysis, and relevant news or events for market impact as data sources. He says with the right data sources, “Businesses can gain a more comprehensive understanding of potential disruptions and trends, improving forecast efficiency as well as strategic decision-making and market positioning.”

Tools for revenue lifecycle management

Data scientists may be enthusiastic about creating machine learning models for revenue forecasting, but this can be a mistake. Many revenue forecasting tools include collaboration, annotation, and advisory features to support an FP&A workflow. Forecasting capabilities like SAP Revenue Growth Optimization, Workday Adaptive Planning, Microsoft Dynamic Sales, and Netsuite plug into enterprise ERP and sales platforms. Revenue lifecycle management capabilities provide input data for revenue forecasting and workflows like quote-to-cash and contract management that help improve forecasting accuracy.

Peterson of Conga says, “An open revenue lifecycle management platform with AI at the foundation and an open unified data solution can transform an organization through AI-driven revenue insights, better collaboration across cross-functional teams involved in the revenue lifecycle, and help facilitate holistic decision-making through a single source of truth.”

Data professionals should learn more about how their organizations forecast revenue and use forecasting analytics to enable strategic decision-making. A good place to start is to review who is responsible for forecasting along with the following:

• The types of forecasts they provide
• The data sets they leverage
• The frequency and schedule of delivering forecasts
• The data quality issues they must address
• The data models they use
• The technologies that are critical to their workflow

Data scientists, engineers, and governance specialists can provide significant value by collaborating on forecasts, centralizing data, improving data quality, and sharing expertise around modeling and visualization.