Stephen J. Thomas, Ph.D. - Scalable Krylov Solvers for GPU-Accelerated

Stephen J. Thomas, Ph.D.

Superior, Colorado (************@*****.***, 720-***-****)

Research focus. Scalable numerical linear algebra and GPU-resident solvers with emphasis on reduced memory-read footprint and synchronization on AMD MI-class accelerators. Current work develops Streaming Krylov Acceleration (SKA) for training and Skyline QR for inference. Both are short-degree polynomial/subspace methods designed for bandwidth-bound regimes on MI300/MI300X and multi-GPU nodes.

Core areas. Krylov methods, algebraic multigrid, Chebyshev/minimax polynomial smoothers, mixed precision and backward stability, s-step and low-synchronization orthogonalization, sparse- kernel layout and read-avoidance streaming Gauss-Seidel, Rayleigh-band analysis for polynomial filters. Stochastic multigrid based on the JL transformation and sketching. Professional experience

Principal Member of Technical Staff, AMD 2022–2025 Mixed-precision AMG and Krylov R&D for MI-series GPUs; solver kernels and precision placement on MI210/MI300-class systems; read-minimizing SpMV and low-synchronization orthogonalization; analysis of V-cycle fidelity with implicit coarse solves. Senior Applied Mathematician, DOE/NREL 2017–2022

Solver lead for exascale CFD/physics applications; AMG/Krylov design and early GPU transitions with Trilinos/hypre; performance modeling under bandwidth constraints. Earlier roles at Argonne MCS, NCAR, Acceleware, and Schlumberger in large-scale simulation and solver development.

Independent research

Founder and Lead, SKA.ai 2023–present

Streaming Krylov Acceleration (training) and Skyline QR (inference). Clean-gradient regime anal- ysis; history Krylov bridge; Chebyshev-on-band contraction; MI300-oriented implementations emphasizing reduced reads and no global reductions in the inner loop. Education

Ph.D., Computer Science, Universit e de Montr eal, 1993 (Krylov subspace projection methods) M.Eng., Electrical Engineering, McGill University, 1986 (generalized-covariance filtering) B.Math (Honours), Applied Mathematics, University of Waterloo, 1983 Recent manuscripts and submissions (AI optimization & inference) The following five papers cover SKA (optimizer) and Skyline QR (inference). Journal targets are per current submission plan.

1. Streaming Krylov Acceleration for Stochastic Optimization: Minimal Theory and Empirical Validation. Submitted to SIAM Journal on Optimization (SIOPT), 2025. 1

2. Low-Rank Krylov Methods for Large-Scale Stochastic Optimization. Submitted to SIAM Journal on Mathematics of Data Science (SIMODS), 2025. 3. Streaming Krylov Acceleration for Neural Network Optimization: Theory, Implementation, and Empirical Validation. Submitted to SIAM Journal on Mathematics of Data Science

(SIMODS), 2025.

4. Adaptive Subspace Projection for Accelerated Inference in Transformer Models (Skyline QR). Target journal: SIAM Journal on Mathematics of Data Science (SIMODS), 2025. 5. Randomized Multigrid Projectors for Gradient Covariance and Two-Grid Training. Target journal: SIAM Journal on Matrix Analysis and Applications (SIMAX), 2025. Additional current papers

s-step Conjugate Gradients with One-Sweep Gauss–Seidel and Stability Guarantees (with P. D’Ambra). Submitted to SIAM Journal on Scientific Computing (SISC), 2025. Copper Mountain 2025 (ETNA special volume) and DD-29 (Milan) 2025 submissions on backward stable iterative coarse solves.

Selected prior publications (HPC / NLA)

Overview and stability of block Gram–Schmidt (with Carson, Lund, Rozloˇzn ık), Linear Algebra and its Applications, 2022.

Low-synchronization Gram–Schmidt and GMRES (with Swirydowicz, Langou, et al.), Numerical Linear Algebra with Applications, 2020.

Iterated Gauss–Seidel GMRES, SIAM Journal on Scientific Computing, 2023. Efficient GMRES+AMG on GPUs: composite smoothers and mixed V-cycles, SIAM Journal on Scientific Computing, 2023.

Research summary for Sandia CSRI

The work is organized around mathematically disciplined reductions in memory traffic and synchro- nization for multilevel linear algebra and AI. For SKA, the reduced step is an H-metric projection in the streamed history subspace; a projector bound shows the step is an O(δ) perturbation of a degree-k−1 polynomial step; Chebyshev minimax on a Rayleigh band yields contraction 2ρ k h up

to a small additive perturbation. For Skyline, stabilized QR maintains an activation subspace on- line so that inference executes in a compact basis with fewer reads. Both methods are tuned for MI300-class memory systems and align with Sandia’s HPC–AI integration agenda. Honors and service

IEEE Gordon Bell Award (Special Category), second place (team). Associate Editor, Monthly Weather Review. Organizer, SIAM PP/CSE minisymposia in sparse solvers and multigrid. Grad- uate and postdoctoral mentoring.

2

Selected earlier publications (Krylov recycling)

Work with A. Baker and collaborators on recycling/augmentation for sequences of linear sys- tems; includes augmented and deflated GMRES, block and shifted variants, and applications to CFD/structural solves. Representative topics and contributions include: recycled subspaces for restarted GMRES; deflation/augmentation equivalence and stability; preconditioned recycling; pa- rameterized and shifted systems; error/residual orthogonality under truncation; complexity models for cycling strategies. (2008–2014; list of specific articles available on request.) Selected earlier publications (hypre / BoomerAMG)

Publications and reports related to the hypre library’s multigrid components, with emphasis on BoomerAMG design and performance at scale. Representative areas: coarsening and interpolation strategies; Chebyshev and polynomial smoothers; mixed-precision relaxation; GPU enablement and data layout for SpMV/SpGEMM; V-cycle fidelity and coarse-grid operator formation; comparisons with Trilinos/ML and GAMG. (2006–2016; list of specific articles available on request.) Software and community contributions

Core and extension contributions to hypre (BoomerAMG) and Trilinos ecosystems; early GPU pathways for smoothers and SpMV/SpGEMM; tutorial and minisymposium organization for SIAM PP/CSE on multigrid and Krylov recycling; mentorship of students and postdocs on AMG/Krylov topics.

Teaching

Adjunct Faculty at Lehigh University, Computer Science, Courses in algorithms and optimizers for artificial intelligence, HPC

References

Available on request. Ray Tuminaro (SNL CA), Eric Cyr (SNL) 3

Contact this candidate