Kamal Ghorui
Many times, the only truth that remains is the effort put into solving a business problem. The outcome eludes us. An honest data scientist would admit. Data science teams have struggled to attribute revenue improvements or cost savings to their work.
The top-down approach of identifying opportunities
In a top-down approach to solving a business problem, one understands the requirements and drills down into the data (primarily tables and columns in a database) to determine whether the data supports the stated problem. If the underlying data has not captured the problem statement (or a possible solution), there is nothing much a data scientist can do. At best, they can recommend starting to capture the missing data and proposing solving the problem in the future (let’s say, a year from now). Usually, this will not align with the leadership team’s short-term goals. However, this must be done anyway, preparing for the future.
The bottom-up approach of identifying opportunities
In a bottom-up approach to identifying hidden opportunities to improve business outcomes, one first studies the available data, spots problems, and proposes solutions. The data science team starts working on solutions only after approval from the business team and after appropriate feasibility checks, expected return on investment (ROI), and prioritisation. This way, the data clearly supports the business problem, since we started with the data and worked our way up. However, as in the earlier case, the outcome could be unreachable.
Making data say what you want to hear
When a data scientist becomes passionate and emotionally attached to the work they are doing, it is easy to end up force-fitting a data solution to the business problem. It’s extremely difficult to admit that our analysis didn’t discover anything new or actionable. This happens when we see a failed project as a “failure” of our capabilities and talent. A data science project increases the team’s familiarity and understanding of the data, regardless of whether the intended objective was achieved. The next time the team works in the same area, they hit the ground running. Don’t make the data say what you want to hear. Instead, listen to what the data says with an open mind.
Capturing additional data
Usually, we miss planning for the exercise of identifying the kind of data to be captured that would help provide an entirely new solution to an existing problem or enhance an existing solution in the future. This item must feature in an organisation’s long-term data strategy. While exclaiming, getting overwhelmed by the current volume of data, and studying the possibilities for what can be done with them, it is easy to miss the item.
Being open to non-actionable outcomes or non-useful data (as we might be tempted to say) is a valuable lesson for any data scientist. While it is easy to advise, as leaders, we must create an ecosystem of service providers, clients, and stakeholders that does not punish those who fail at “data experiments”. We must be open to resetting and restarting our thinking, projects, or strategies whenever the situation demands it. If it helps, ask any data scientist with 15 years of experience how many successful projects they have implemented in production (not CV-wise).
In the end, data science is less about guaranteed outcomes and more about disciplined inquiry. Not every project will yield measurable ROI, and not every dataset will reveal a hidden growth lever. Yet, each attempt sharpens intuition, exposes data gaps, and strengthens future decision-making. The real maturity of a data-driven organisation lies not in celebrating only successes, but in valuing honest exploration and learning. When teams stop chasing validation and start embracing uncertainty, they move closer to meaningful impact.
END OF ARTICLE
