Splunk Engineer

Job Title:
Splunk Engineer
Duration: 6 Months

Location:

Phoenix, AZ

About the position
Onboard new systems and data sources into the City's enterprise Splunk Cloud environment and developing dashboards, alerts, and analytics to improve operational visibility, security posture, and service reliability. This role is highly technical and requires strong expertise in Splunk administration, data onboarding, and SPL (Search Processing Language) development.

Key Responsibilities

• Onboard new systems, logs, and data sources into Splunk, ensuring proper parsing, field extractions, CIM compliance, and data normalization.
• Configure and maintain forwarders, ingestion pipelines, and data routing.
• Build advanced dashboards, visualizations, and analytics for operational, security, and business use cases.
• Develop complex SPL queries, macros, lookups, and scheduled searches.
• Troubleshoot ingestion issues, search performance, and data quality problems.
• Partner with network, server, application, and security teams to define log requirements and actionable monitoring.

• Experience administering and engineering Splunk Enterprise or Splunk Cloud in a medium-to-large environment.
• Strong proficiency with SPL for analytics and troubleshooting.
• Demonstrated experience onboarding new systems or applications into Splunk.
• Experience building dashboards with Splunk Dashboard Studio or Classic Editor.
• Knowledge of log ingestion formats (syslog, JSON, XML), data parsing, and field extraction.
• Understanding of IT infrastructure fundamentals (servers, networks, firewalls, cloud services).
• Experience with Linux command line and Splunk Universal/Heavy Forwarder management.

• Experience with automation or scripting (Python, Power Shell).
• Experience with Enterprise Security (ES) or ITSI modules.
• Familiarity with indexer clustering, search head clustering, and distributed Splunk architectures.
• Experience implementing CIM compliance and data models.

• Strong analytical, troubleshooting, and visualization skills.
• Ability to work collaboratively with infrastructure, application, and security teams.
• Strong communication skills and the ability to translate technical findings into actionable insights.