Welcome to xFiles, Solver's investigative video series that uncovers the most perplexing mysteries plaguing finance teams everywhere. Each episode solves common financial planning nightmares that keep CFOs awake at night.
A longtime NetSuite customer came to Solver with a problem that sounded almost impossible to fix: their data integration was running 20 hours a day. That left a four-hour window, at most, to actually report on anything. And a process that runs for 20 hours straight is exactly the kind of process that's likely to fail somewhere along the way, which it often did, taking that day's data down with it.
Every failed run meant lost data and a reporting window that kept shrinking. The team wasn't just fighting a slow integration. They were fighting the clock, every single day.
Solver investigator Chase Sariaslani opened the case with a simple question: why does this integration take 20 hours in the first place? The answer was structural. The integration was pulling all-time-to-date data, every record, every time, instead of just what was new. Adding a period filter looked like the obvious fix, and the case looked closed.
It wasn't. The person who'd originally built the integration no longer worked at the company, and once Chase dug in, there was a reason the filter had never been added in the first place: the posting period field the customer needed to filter on wasn't actually filterable through standard means.
Watch the full investigation:
The fix took three parts, and each one uncovered another layer of the mystery.
Part 1: Add the period filter. The team narrowed the pull from all-time-to-date down to a defined window, the last 18 months, based on what the customer actually needed.
Part 2: Build a custom RESTlet. Because the standard fields wouldn't support filtering on posting period, Solver's development team built a new RESTlet script that could filter on exactly the field the customer needed. That single change brought the integration down from 20 hours to an hour and a half.
Part 3: Fix the rules, not just the integration. The customer's rules were still running an old all-time-to-date process left over from their original implementation. Updating those rules brought the entire job down to just 30 minutes.
The result? An integration that used to eat 20 hours a day now runs in half an hour. The customer can run it daily, or on demand, even in the middle of month-end close, instead of waiting on a single overnight window. Data integrity improved, errors dropped, and the customer, in their own words, was happy as a clam.
If your integrations are eating your reporting window, or your close, it's worth asking whether they're pulling more data than they actually need. Take a closer look at how a data warehouse built for unlimited sources handles integrations that are built to run in minutes instead of hours.