Production Systems Engineer, AI Systems - Meta

Meta is seeking a Systems Engineer to join our Release to Production (RTP) team working on Meta Training and Inference Accelerator (MTIA) program as a part of the AI/ML initiatives supporting large scale AI Training and Inference. Our servers and data centers are the foundation upon which our rapidly scaling infrastructure operates efficiently to deliver our innovative services. The RTP team is responsible for the end-to-end Hardware Lifecycle of all Meta servers, including prototyping of experimental HW, pre-production hands-on system and hardware debugging and stress testing, enabling production-ready system monitoring, automated provisioning and automated remediation of issues. RTP team also helps in exploring, developing and productizing high-performance software and hardware technologies for AI at datacenter scale.RTP Engineers have a large swath of cross-functional partners they work closely with e.g. HW/SW co-design teams, hardware designers, networking teams, system manufacturers, component vendors, capacity engineering, production engineering, production services, and data center operations teams to enable new systems that will be deployed in our production data centers. We are looking for a candidate to work on scale up and scale out network technologies (e.g RDMA NIC) for Meta Training and Inference Accelerator (MTIA) systems that are powering Meta’s tremendous leaps in the AI space. The ideal candidate is knowledgeable about network protocols (TCP/IP, RDMA) and has hands-on experience driving post-Silicon validation for networking platforms, all the way to mass production and deployment.

RESPONSIBILITIES

Production Systems Engineer, AI Systems Responsibilities:

• Support new MTIA platform introduction into Meta fleet by working closely with post-silicon validation team to successfully test network interfaces at protocol/system level, as well as owning the network interface system validation all the way to mass production
• Proactively create experiments and tooling to detect, reproduce and diagnose hardware/firmware/software health issues
• Develop understanding of AI workload traffic and incorporate as part of NPI
• Contribute to enabling hacks for future technology explorations in AI space
• Troubleshoot, diagnose and root cause of system failures and isolate the components/failure scenarios while working with internal & external partners
• Develop visibility through data visualization and implement systemic solutions to hardware health issues
• Leverage production experience to drive external and internal teams to continuously improve product quality

MINIMUM QUALIFICATIONS

Minimum Qualifications:

• Bachelor’s degree in Engineering or Computer Science
• 6+ years of work experience in one or more domains such as: Network ASIC development (Silicon design or bring up or characterization), Network Product deployment and customer support (Switches, NICs), Interconnect Technologies (e.g. Optics, DAC)
• Knowledge of server architecture and components
• Experience working with Linux
• Knowledge of TCP/IP and experience using iperf
• Hands on troubleshooting and debug experience

PREFERRED QUALIFICATIONS

Preferred Qualifications:

• Experience working with Network Interface Cards (NICs)
• Experience working with RDMA/RoCE
• Experience working with full server system, including PCIe
• Experience working with large scale deployments