Senior GPU Cloud Storage Solutions Engineer (SRE SME)

About the role

You will own the high‑performance I/O layer that powers AI model training on Bitdeer’s GPU cloud. The role focuses on preventing storage‑related faults that can stall multi‑hundred‑GPU jobs, using AIOps signals to predict and remediate issues before they impact costly training runs.

Key responsibilities

• Deploy and operate parallel/distributed storage systems such as WEKA, VAST Data, Ceph and DDN/Lustre.
• Design storage architectures optimized for AI workloads, including checkpoint I/O bursts, sequential dataset reads and KV caches for inference.
• Implement multi‑tenant storage isolation with per‑tenant QoS, quotas and access controls; configure GPU Direct Storage for direct GPU‑to‑storage paths.
• Manage storage networking (NFS over RDMA, NVMe‑oF, high‑speed fabrics) and Nvidia CMX for cluster‑wide orchestration.
• Diagnose and tune storage performance (IOPS, throughput, latency) using tools like fio, IOR and mdtest; maintain runbooks for common failure modes.
• Plan capacity aligned with GPU cluster growth, handling firmware updates, data migration and disaster‑recovery procedures.
• Instrument storage telemetry (IO tail latency, checkpoint duration, NVMe SMART, filesystem health, RDMA counters) for the AIOps platform.
• Collaborate with the platform team to define and improve storage‑fault prediction models.

Required profile

• Senior‑level experience in designing, operating and troubleshooting high‑performance storage systems for AI or HPC workloads.
• Strong understanding of GPU‑direct storage pathways and storage networking technologies.
• Proven ability to work in a fast‑paced, multi‑tenant cloud environment and to drive capacity planning.

Required skills

• WEKA
• VAST Data
• Ceph
• DDN/Lustre
• GPU Direct Storage
• NFS over RDMA
• NVMe‑oF
• Nvidia CMX
• fio
• IOR
• mdtest