Software Engineer Forecast Engine

Job Description

Employees can work remotely.

Join the Global Cloud Services organization's Fin Ops Tools team, which is building Service Now's next-generation analytics and financial governance platform. Our team owns the full modern data stack:
Trino for distributed queries, dbt for transformations, Iceberg for lakehouse architecture, Lightdash for business intelligence, and Argo Workflows for orchestration. You will own the Forecast Engine, the system that turns Service Now's cloud capacity and cost actuals into forward-looking forecasts, then automatically tracks those forecasts against plan and budget and alerts the right people when reality diverges. The Forecast Engine also feeds directly into our Future Capacity Reservation (FCR) automation: its forecast of fleet growth and workload migration timing is the signal that drives how much hyperscaler capacity to reserve, in which providers and regions, and when, against the lead-time windows Fin Ops and Cloud Operations plan around.

The Forecast Engine is the simulation and automation core behind Fin Ops capacity and cost planning. It reads forecasting actuals from the lakehouse and runs a deterministic multi-period simulation of fleet growth, workload migration, placement, and sizing. It validates each result against hard in variants and publishes forecasts that data scientists, analysts, and Fin Ops engineers consume in Lightdash. Today it is a fast, single-binary Rust core with a streaming Trino read and an Iceberg publish path.

The next chapter is to turn that engine into an automated, always-on forecasting service.

• Design and develop scalable, maintainable, and reusable software components with a strong emphasis on performance, determinism, and reliability.
• Collaborate with product managers and Fin Ops partners to translate planning and budgeting requirements into well-architected solutions, owning features from design through delivery.
• Build intuitive and extensible interfaces for forecast consumption (Lightdash models, alert payloads, and APIs) ensuring flexibility for finance and capacity-planning use cases.
• Contribute to the design and implementation of new Forecast Engine capabilities while enhancing existing simulation, validation, and publish paths.
• Integrate automated testing into development workflows to ensure consistent quality across releases, including determinism (byte-identical output) and forecast-accuracy regression checks.
• Participate in design and code reviews ensuring best practices in performance, maintainability, and testability.
• Develop comprehensive test strategies covering functional, regression, integration, and accuracy aspects (period-over-period identity, backtest grading against real actuals).
• Foster a culture of continuous learning and improvement by sharing best practices in engineering and quality.
• Promote a culture of engineering craftsmanship, knowledge-sharing, and thoughtful quality practices across the team.

• Own the architecture of the Forecast Engine and the automation layer around it: scheduled runs, variance/budget tracking, and alerting.
• Lead technical decision-making on forecast cadence, reconciliation against actuals, alert routing, and the contract between the simulation core and downstream consumers.
• Establish best practices for forecast automation: idempotent scheduled runs, deterministic reproducibility, fail-loud data contracts, and no silent fallbacks.
• Define how forecast signals (variance, budget breach, capacity headroom, migration drift) are computed, threshold ed, and surfaced.
• Drive innovation in forecasting and planning automation, including the responsible use of AI/ML tooling to accelerate development and analysis.

• Build the automation that runs the Forecast Engine on a schedule via Argo Workflows, with retries, alerting on failure, and run-to-run reproducibility.
• Develop variance and budget tracking: reconcile each forecast against plan and against the latest actuals, compute deltas at the grains that matter (provider, region, pod, workload), and persist a queryable variance history.
• Implement alerting that fires on budget breach, forecast drift, capacity thresholds, and pipeline health, routed to Splunk and the team's notification channels.
• Integrate with planning systems so plan/budget targets flow into the engine and forecast outputs flow back out to the planning surface.
• Drive the Future Capacity Reservation (FCR) handoff: translate the forecast of fleet growth and migration timing into reservation recommendations (how much capacity, which providers/regions/pods, and by when), aligned to hyperscaler procurement lead-time windows and reconciled with Cloud Operations so the same capacity is never reserved twice.
• Build and extend the Rust simulation core (period loop, growth, migration, routing, packing, sizing, validation) and its streaming Trino read and Iceberg publish paths.
• Create and maintain the Lightdash…