Data​/Systems Engineer; Bid - TS​/SCI w​/Poly

The Sponsor requires Data Engineering support to evaluate, optimize, and implement robust data infrastructure that enables reliable, accessible, and scalable data delivery across the organization. The Contractor will work collaboratively with data consumers, technical teams, leadership, and stakeholders to assess current data pipelines, identify gaps in data accessibility and reliability, and architect solutions that establish trusted data foundations. Work involves applying engineering best practices to implement proper data modeling and integration patterns, ensuring data quality and observability throughout pipelines, and creating maintainable infrastructure that supports analytics, reporting, and operational use cases.

The Sponsor’s data landscape includes enterprise operational systems such as Service Now, network management platforms (NetIM), and network modeling tools (Forward Networks). The Data Engineering support must be adept at extracting data from these systems via APIs (application Programming Interface), exports, and vendor-specific interfaces, often with limited documentation or non-standard data structures, and transforming this operational data into accessible, integrated datasets.

The Systems Engineer will conduct comprehensive assessments of existing data pipelines, infrastructure, and data flows including integrations with operational systems like Service Now, network management platforms, and business applications to identify technical debt, bottlenecks, and reliability issues. The Systems Engineer will evaluate current data architecture against industry best practices and organizational needs; develop technical recommendations and roadmaps for data infrastructure improvements.

The Systems Engineer will design, build, and maintain production-grade data pipelines using orchestration tools such as Airflow or Prefect. The Systems Engineer will develop robust ETL(Extract-Transform-Load/ELT (Extract-Load Transform) processes from diverse sources:
SaaS platforms, network management systems, databases, APIs, files, and streams. The Systems Engineer will build API integrations handling authentication (OAuth, API keys, and Single Sign-On (SSO), rate limiting, pagination, retry logic, and error handling). The Systems Engineer will extract data from systems not designed for export; reverse-engineer undocumented data structures and relationships. The Systems Engineer will handle semi-structured data (JSON and XML);

and transform into structured datasets with consistent schemas. The Systems Engineer will design dimensional models, data warehouses, and data marts following industry methodologies. The Systems Engineer will create conceptual, logical, and physical data models optimized for query performance and storage efficiency. The Systems Engineer will implement slowly changing dimensions and other data warehousing patterns. The Systems Engineer will establish naming conventions, data standards, and modeling best practices.

The Systems Engineer will implement comprehensive data quality checks, validation rules, and automated monitoring with alerting. The Systems Engineer will build error handling, failure recovery, logging, and observability into all processes. The Systems Engineer will optimize pipelines for performance, cost, and resource utilization. The Systems Engineer will develop reusable components and frameworks; refactor legacy pipelines for reliability. The Systems Engineer will build and maintain data infrastructure on cloud platforms (Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP) using infrastructure-as-code using Terraform and Cloud Formation.

The Systems Engineer will implement CI/CD pipelines, version control (Git), and automated testing frameworks. The Systems Engineer will manage database performance tuning, indexing, partitioning, and capacity planning. The Systems Engineer will establish backup, recovery, security controls, access controls, and compliance measures. The Systems Engineer will partner with analysts, software developers, and business stakeholders to translate requirements into technical solutions. The Systems Engineer will create comprehensive documentation for systems, processes, and integrations.

The Systems Engineer will provide technical guidance on data availability and proper usage; enable self-service access. The Systems Engineer will troubleshoot pipeline failures, performance issues, and data discrepancies; perform root cause analysis.

• (Mandatory) Demonstrated experience designing, building, and maintaining production data pipelines using orchestration tools such as Apache Airflow or similar.
• (Mandatory) Demonstrated experience with SQL skills including complex queries, optimization, and performance tuning across multiple database platforms.
• (Mandatory) Demonstrated experience integrating data from Sponsor SaaS platforms and operational systems via APIs, including handling authentication, pagination, and rate…