Senior Principal Software Engineer (Hybrid) - District Partners LLC

We are a technology-driven organization focused on delivering innovative solutions to support mission-critical cyber operations. We pride ourselves on recruiting top engineering talent and tackling complex problems that require creativity and expertise. Our success is measured by the impact and effectiveness of the solutions we deliver.

Position Overview:

We are seeking an experienced software engineer who thrives at the intersection of system operations and software functionality. The ideal candidate will be self-driven, able to troubleshoot independently, and comfortable collaborating with diverse specialists. You’ll be responsible for ensuring the smooth operation of services, identifying issues, and communicating solutions effectively.

A strong understanding of Linux administration, DNS, certificates, and containerized environments, along with intermediate experience with Kubernetes, will be essential for success in this role.

Key Responsibilities:

• Collaborate with team members to resolve complex system issues
• Enforce development, code, and testing standards
• Troubleshoot and maintain both off-the-shelf and custom software services
• Address user-reported issues promptly and effectively
• Develop processes for maintaining and patching live services
• Manage automated backups of service data and monitor systems for any anomalies
• Plan and execute version upgrades, including database migrations and backup/restore procedures
• Work closely with service development teams to streamline deployments and improve service automation
• Communicate clearly with both technical teams and end-users during system upgrades or maintenance
• Document detailed logs and procedures for all maintenance activities
• Plan and schedule security scans and system updates to maintain compliance
• Participate in team planning and offer recommendations for service improvements
• Review and validate code and deployment changes as part of the quality control process

Required Skills & Experience:

• Active US TS/SCI clearance
• Bachelor's degree in Computer Science, Software Engineering, or a related field (may be substituted with additional hands-on experience)
• Minimum of 10 years of industry experience in DevOps, software engineering, or similar roles
• Proficiency in container technologies like Docker or Podman and orchestration tools like Kubernetes
• Hands-on experience with Infrastructure-as-Code (e.g., Terraform)
• Strong Linux development environment experience
• Familiarity with relational databases (e.g., PostgreSQL)
• Knowledge of cloud infrastructure, with Azure experience being a plus
• Understanding of networking principles, DNS, proxies, and certificates
• Familiarity with GitLab and CI/CD processes
• Proficiency in Python programming

Preferred Skills:

• More than 5 years in a DevOps or DevSecOps role
• Experience managing Kubernetes clusters and deploying applications within those environments
• Familiarity with container security practices, including building, scanning, and remediating container images
• Strong communication skills for working with clients and end-users

We’re looking for an engineer who can seamlessly blend problem-solving skills with a passion for continuous learning and operational excellence. If you're excited about working in a dynamic, high-trust environment, we’d love to hear from you!